Create X86

Create X86 GROMACS cluster

We will reuse the guided and refined configuration we just created to spin up a X86 cluster configuration.

read -p "Name of the cluster you want to create: " CLUSTER_NAME

Let us commit this to our environment variables and start refine the config.

echo "export CLUSTER_NAME=${CLUSTER_NAME}"| tee -a ~/.bashrc
cp pcluster-base.ini ${CLUSTER_NAME}.ini

Adjust Configuration

Instance Types

In our base config we picked c5n.18xlarge as compute and m5.2xlarge as head node. Let us add g4dn instances.

C5

cat ${CLUSTER_NAME}.ini \
    |wildq -i ini -M '."queue c5".disable_hyperthreading = "false"' \
    |wildq -i ini -M '."queue c5".enable_efa = "false"' \
    |wildq -i ini -M '."queue c5".compute_resource_settings = "c5"' \
    |wildq -i ini -M '."compute_resource c5".instance_type = "c5.24xlarge"' \
    |wildq -i ini -M '."compute_resource c5".max_count = "2"' \
    |sponge ${CLUSTER_NAME}.ini

C5n

cat ${CLUSTER_NAME}.ini \
    |wildq -i ini -M '."queue c5n".disable_hyperthreading = "false"' \
    |wildq -i ini -M '."queue c5n".enable_efa = "true"' \
    |wildq -i ini -M '."queue c5n".compute_resource_settings = "c5n"' \
    |wildq -i ini -M '."compute_resource c5n".instance_type = "c5n.18xlarge"' \
    |wildq -i ini -M '."compute_resource c5n".max_count = "8"' \
    |sponge ${CLUSTER_NAME}.ini

G4dn

For single-node jobs, g4dn instances with a single GPU have a compelling price/performance ratio.

cat ${CLUSTER_NAME}.ini \
    |wildq -i ini -M '."queue g4dn".disable_hyperthreading = "false"' \
    |wildq -i ini -M '."queue g4dn".enable_efa_gdr = "false"' \
    |wildq -i ini -M '."queue g4dn".compute_resource_settings = "g4dn"' \
    |wildq -i ini -M '."compute_resource g4dn".instance_type = "g4dn.8xlarge"' \
    |wildq -i ini -M '."compute_resource g4dn".max_count = "3"' \
    |sponge ${CLUSTER_NAME}.ini

Update queue_settings

The global cluster setting defines which queue/compute configurations to pick up.

cat ${CLUSTER_NAME}.ini \
    |wildq -i ini -M '."cluster default".queue_settings = "c5n,c5,g4dn"' \
    |sponge ${CLUSTER_NAME}.ini

Create

After the refinement we’ll kick of the creation. We will use the following flags:

  • -c <file> defines the configuration file we just created;
  • --nowait will kick off the deployment and detach;
  • --norollback won’t roll back in case there is an error. That is helpful for debugging your post-install scripts.
pcluster create -c ${CLUSTER_NAME}.ini ${CLUSTER_NAME} --nowait --norollback

We are using the following command to attach to the status:

pcluster status ${CLUSTER_NAME}

After a couple of minutes (~10min) you will see pcluster finish.