Slurm Alloc

Before we submit jobs to the scheduler, we are going to run them interactively.

SLURM

First we allocate three nodes from the head node.

salloc -N3 --exclusive

Once the nodes are ready we are going to have a bash with the environment set up. To carry over the environment variables, which SLURM provides, we use a small script.

cat > slurm-ssh.sh << \EOF
#!/bin/bash
# Usage: salloc <options> slurm-ssh
first=$(scontrol show hostname $SLURM_NODELIST | head -n1)
env=$(printenv | grep SLURM | sed -rn "s/=(.*)/='\1'/p" | paste -d' ' -s)
exec ssh $first -t "$env zsh"
EOF
chmod +x slurm-ssh.sh
./slurm-ssh.sh

MPIRUN

To run gromacs we are going to load the spack modules gromacs.

C5n

spack load gromacs@2020.5%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
time mpirun -n 18 gmx_mpi mdrun -ntomp 4 -s /fsx/input/gromacs/benchRIB.tpr -resethway

C5n

spack load gromacs@2020.5%gcc@7.3.1 arch=linux-amzn2-zen
time mpirun -n 48 gmx_mpi mdrun -ntomp 2 -s /fsx/input/gromacs/benchRIB.tpr -resethway