Slurm

Revision as of 13:23, 23 September 2021

The Slurm job scheduler on the High Performance WiWi Cluster (HPC3)

1 Introduction

2 Cluster topology & hardware specs

3 Submitting jobs

3.1 Batch jobs

Slurm provides support for unattended execution of jobs on the cluster's resources , which is perhaps the most common way of using it (batch mode). For this purpose, a shell script is passed to the job scheduler, containing

• the commands to be executed and
• some extra information for the slurm job scheduler (optional).

Let us take a closer look at how to create such a script. We start with the first line, telling the OS which kind of UNIX shell to use for interpreting the commands in the script.

#!/bin/bash

Then we add a series of directives for the slurm job scheduler, each starting with a '#SBATCH'. Although the '#' character usually indicates a comment, this specific string gets interpreted by slurm and allows to set various options.

#SBATCH --mail-type=BEGIN,END
#SBATCH --mail-user=john.doe@univie.ac.at

For the moment, we only state an e-mail address here and an indication which events trigger a notification via mail. In this case, we receive an e-mail when the job has been started, that is, when it is removed from the queue of waiting jobs and actually allocates resources on the cluster.

Finally, we add commands to be executed for actual computation purposes. Let us assume in the following that the program we would like to run is called do-something, allowing single- or multi-threaded execution. Assume further that threading can be controlled by a command line parameter --threads. If we wanted to use all 16 or 32 processors of a standard allocation (1 socket), then the program could be run either by

do-something --threads 16

or by parallelizing single-threaded instances of itself:

do-something --threads 1 &
do-something --threads 1 &
...
do-something --threads 1 &

Note that the '&' character at the end of each line tells the shell to run the program in background mode. The second mode of execution is useful, for example, when each instance of do-something takes a different file as an input.

When saving the script to the disk as a file, for example job-script.sh, we can run it using the sbatch command:

sbatch -J Job1 job-script.sh

The command takes the contents of the file job-script.sh, and tries to allocate resources on the cluster. If there are enough resources available (at least one socket) then the job is started on the corresponding node. Otherwise the job is held in the queue. Parameter '-J' assigns a name to the job.

#SBATCH --partition=apollo
#SBATCH --qos=normal

3.2 Interactive jobs