ParallelCluster

In this chapter we are going to install and configure ParallelCluster in the Cloud9 environment. ParallelCluster is installed using python-pip.

This script will be executed every time a new terminal (CLoudShell) is opened. With CloudShell it might happen that a new instance is used and we need to reinstall.

cat > ${HOME}/pcluster-install.sh << EOF
#!/bin/bash

if ! command -v pcluster &> /dev/null
then
  echo ">> pcluster is missing, reinstalling it"
  sudo pip3 install aws-parallelcluster
else
  echo ">> Pcluster $(pcluster version) found, nothing to install"
fi
EOF
chmod +x ${HOME}/pcluster-install.sh
echo "bash ${HOME}/pcluster-install.sh" >> ~/.bashrc
bash ${HOME}/pcluster-install.sh

This will result in the following output:

Tools

We’ll use a couple of tools to make life easier.

  • moreutils: The command sponge allows us to read and write to the same file (cat a.txt|sponge a.txt)
  • jq: Manipulating json
  • wildq: jq on steroides to manipulate INI files
cat > ${HOME}/prereq-install.sh << EOF
if ! command -v sponge &> /dev/null
then
  echo ">> sponge is missing, reinstalling prerequisites"
  sudo amazon-linux-extras install epel -y
  sudo yum --enablerepo epel install -y moreutils
  sudo yum install -y jq
  sudo pip3 install wildq
else
  echo ">> prerequisites are installed, nothing to do"
fi
EOF
chmod +x ${HOME}/prereq-install.sh
echo "bash ${HOME}/prereq-install.sh" >> ~/.bashrc
bash ${HOME}/prereq-install.sh