cluster-env (Shell Settings System)¶

How users' shell settings are delivered from one shared place. Introduced September 2026. See Cluster Conventions for where this fits into the broader /SLURM/public layout.

Why¶

• salloc shells (with use_interactive_step) are non-login shells, so /etc/profile.d/lmod.sh didn't run and module was missing inside jobs.
• Settings (conda init, package cache, SQUEUE_FORMAT) were copied into every user's .bashrc over the years, in several versions.
• Node-local hooks (/etc/profile.d, /etc/bash.bashrc) drift between nodes and miss nodes that are down.

Result: one loader on NFS, one small file per feature, and one visible line per feature in each user's .bashrc.

Files¶

/SLURM/public/etc/
├── cluster-env.sh            # loader: defines cluster_env; loads nothing by itself
└── cluster-env.d/
    ├── 00-module.sh          # module
    ├── 05-slurm.sh           # slurm
    ├── 20-conda.sh           # conda
    ├── 30-nexus.sh           # nexus
    └── 40-jupyter.sh         # jupyter

All root-owned, mode 644, directories 755 (chmod -R a+rX,go-w).

How It Runs¶

In the user's ~/.bashrc (top of file, before the interactive check):

# >>> JRCAI cluster settings (comment a line to turn it off) >>>
source /SLURM/public/etc/cluster-env.sh
cluster_env module        # module command (Lmod)
cluster_env slurm         # SLURM defaults
cluster_env jupyter       # jupyter-start command
cluster_env conda         # Anaconda
cluster_env nexus         # JRCAI package cache
# module load ollama      # Ollama
# <<< JRCAI cluster settings <<<

1. source cluster-env.sh defines cluster_env (nothing else is loaded).
2. Each cluster_env <name> finds cluster-env.d/NN-<name>.sh and sources it in the current shell.
3. Because the block is above case $- in *i*) ...; return, it runs in every bash that reads .bashrc — interactive shells, salloc shells, ssh host cmd, and batch jobs that source ~/.bashrc.
4. Everything below the interactive check (prompt, aliases, users' own lines) runs only in interactive shells.

Login shells get there too: ~/.profile sources ~/.bashrc (Ubuntu default).

The Loader¶

# JRCAI cluster environment loader.
# Sourced from each user's ~/.bashrc (see /SLURM/public/etc/README.md).
# Loads nothing by itself; each feature is turned on by a line in ~/.bashrc:
#   cluster_env <name>     turn a feature on
#   cluster_env list       show all features and which are on

CLUSTER_ENV_DIR=/SLURM/public/etc/cluster-env.d
CLUSTER_ENV_LOADED=""

_cluster_env_file() {
    local f
    for f in "$CLUSTER_ENV_DIR"/[0-9][0-9]-"$1".sh; do
        [ -r "$f" ] && { echo "$f"; return 0; }
    done
    return 1
}

cluster_env() {
    local f n s
    case "$1" in
        ""|list)
            echo "Cluster environment features:"
            for f in "$CLUSTER_ENV_DIR"/[0-9][0-9]-*.sh; do
                [ -r "$f" ] || continue
                n=$(basename "$f" .sh); n=${n#[0-9][0-9]-}
                case " $CLUSTER_ENV_LOADED " in *" $n "*) s="on" ;; *) s="off" ;; esac
                printf "  %-10s %s\n" "$n" "$s"
            done
            ;;
        *)
            case " $CLUSTER_ENV_LOADED " in *" $1 "*) return 0 ;; esac
            if f=$(_cluster_env_file "$1"); then
                . "$f"
                CLUSTER_ENV_LOADED="$CLUSTER_ENV_LOADED $1"
            else
                case $- in *i*) echo "cluster_env: unknown feature '$1' (see: cluster_env list)" >&2 ;; esac
                return 1
            fi
            ;;
    esac
}

Behaviour:

CLUSTER_ENV_LOADED is not exported: a new shell re-reads .bashrc and reloads everything (needed, because functions and aliases aren't inherited).

Feature Files¶

# Make the "module" command available (Lmod)
if ! type module >/dev/null 2>&1 && [ -f /etc/profile.d/lmod.sh ]; then
    . /etc/profile.d/lmod.sh
fi

export SQUEUE_FORMAT="%.8i %.20j %.10P %.20u %.20a %.8t %.10M %R"

# Optional: Anaconda initialization (also works in batch jobs)
__conda_setup="$('/opt/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
elif [ -f "/opt/anaconda3/etc/profile.d/conda.sh" ]; then
    . "/opt/anaconda3/etc/profile.d/conda.sh"
else
    export PATH="/opt/anaconda3/bin:$PATH"
fi
unset __conda_setup

export PATH="/SLURM/public/apps/jupyter-start:$PATH"

30-nexus.sh (summary):

• Exports PIP_INDEX_URL, PIP_TRUSTED_HOST, PIP_EXTRA_INDEX_URL, PIP_DEFAULT_TIMEOUT=1600, UV_INDEX_URL (Nexus proxy).
• Inside case $- in *i*) ... ;; esac (interactive only): nexus-help, and pip() / uv() wrappers that print a one-line notice on install and call command pip / command uv.

Admin Tasks¶

Change What a Feature Does¶

Edit its file in cluster-env.d/, run bash -n on it. Takes effect in every new shell on every node — no user files change.

Add a Feature¶

1. Create cluster-env.d/NN-<name>.sh (root, 644); put interactive-only parts inside case $- in *i*) ... ;; esac.
2. bash -n it; test with the test user (add cluster_env <name>, open a new shell, cluster_env list).
3. Default on → add the line to /etc/skel/.bashrc and roll it into existing files. Default off → add it commented in the skel only.

Remove a Feature¶

1. Remove the line from /etc/skel/.bashrc and from users' files (sed -i.bak-<name> '/^cluster_env <name>/d' ...).
2. Then delete or empty the feature file. If the file is deleted first, remaining lines only print a warning in interactive shells.

Rename a Feature¶

Add the new file, roll in the new line, remove the old line, then remove the old file.

Checks¶

# syntax of the loader and all features
for f in /SLURM/public/etc/cluster-env.sh /SLURM/public/etc/cluster-env.d/*.sh; do bash -n "$f" && echo "OK $f"; done

# which users have a given feature on
sudo grep -l "^cluster_env nexus" /SLURM/home/*/.bashrc | wc -l

# users without the JRCAI block
sudo grep -L "cluster-env.sh" /SLURM/home/*/.bashrc

# all .bashrc files parse
sudo bash -c 'for f in /SLURM/home/*/.bashrc; do bash -n "$f" 2>/dev/null || echo "SYNTAX ERROR: $f"; done; echo done'

Troubleshooting¶

Design Decisions¶

• No forced/core features — every feature is a visible line users can comment out.
• No node-level hook — loading comes from users' .bashrc (NFS homes), so no per-node files to keep in sync.
• Software is not a feature — software is delivered as modules (module load); cluster_env is for shell settings and small helper commands.
• Root-owned, read-only to users — users can't change shared settings, only turn them on/off.