Post-Install Spack

Next, we create a post-installation script to be executed by ParallelCluster when provisioning the instances.

mkdir -p ~/environment/post-install
cd ~/environment/post-install
cat > 02.install.spack.all.sh << \EOF
#!/bin/bash

### Install yq
yum install -y wget
if [[ $(uname -m) == "aarch64" ]];then 
   wget -qO /usr/bin/yq https://github.com/mikefarah/yq/releases/download/3.4.1/yq_linux_arm64
else
   wget -qO /usr/bin/yq https://github.com/mikefarah/yq/releases/download/3.4.1/yq_linux_amd64
fi
chmod +x /usr/bin/yq
#########
### Install Spack
yum install -y python3
pip3 install boto3
# Install the latest Spack release
export SPACK_ROOT=/usr/local/spack
mkdir -p $SPACK_ROOT
cd $SPACK_ROOT/..
git clone https://github.com/spack/spack
cd $SPACK_ROOT
#git checkout releases/$( \
#    git branch -a --list '*releases*'| \
#    awk -F '/' 'END{print $NF}')
git checkout develop
source $SPACK_ROOT/share/spack/setup-env.sh

echo "export SPACK_ROOT=/share" > /etc/profile.d/spack.sh
echo "source $SPACK_ROOT/share/spack/setup-env.sh" >> /etc/profile.d/spack.sh

cat> /etc/profile.d/spack-env.sh << END
# Make the spack command available to the shell
source $SPACK_ROOT/share/spack/setup-env.sh
END
mkdir -p $SPACK_ROOT/var/spack/environments/aws
### Download aws env file
wget https://gist.githubusercontent.com/bollig/71383f92143ed6b006e5c3892343fef8/raw/2_spack.yaml -O $SPACK_ROOT/var/spack/environments/aws/spack.yaml
### Usersetup
source /etc/os-release 
mkdir -p /fsx/spack
if [[ $ID == "centos" ]];then
  chown -R centos: /usr/local/spack /fsx/spack
elif [[ $ID == "amzn" ]];then
  chown -R ec2-user: /usr/local/spack /fsx/spack
fi

### Change config
export SPACK_SHARED=/shared/spack
yq w -i /usr/local/spack/etc/spack/defaults/config.yaml config.module_roots.lmod ${SPACK_SHARED}/lmod
yq w -i /usr/local/spack/etc/spack/defaults/config.yaml config.module_roots.tcl ${SPACK_SHARED}/modules
yq w -i /usr/local/spack/etc/spack/defaults/config.yaml config.install_tree.root ${SPACK_SHARED}/opt/spack
EOF
chmod +x 02.install.spack.all.sh
aws s3 cp 02.install.spack.all.sh s3://${BUCKET_NAME}/scripts/

Before continuing, check the content of your bucket using the AWS CLI with the command aws s3 ls s3://${BUCKET_NAME} or the AWS console.