Apptainer
Apptainer is a container platform that allows users to have full control of the environment in which their application runs. It is a safe and secure alternative to Docker, as Docker requires root access for much of its functionality, whereas Apptainer allows the safe running of application bundles without requiring root privileges. It is even possible to convert Docker images to Apptainer.
Our public computing systems offer Apptainer at this time in order to enable the portability of highly specialized applications without some of the risks involved with other solutions such as Docker.
In order to use Apptainer, you can either create an image of your application or download it from a hub. Your custom image must be created in either a Linux, Mac, or Windows environment, and then can be moved to any system on which Apptainer is installed, such as our public computing systems, where it can be executed without administrative privileges.
Getting help
The most up-to-date help on Apptainer comes from the command itself:
Apptainer help
The user guide and other helpful information can be found at Apptainer.org.
Apptainer setup
No setup is necessary. Apptainer is installed on all CS clusters.
You can download (pull) a container from a hub. One popular hub is Docker Hub. You can search for a container that fits your requirements.
Pull a container from a hub
Docker Hub
Apptainer can be used to pull docker images from the hub. Docker is not installed on any of the CS servers, so you would have to either run it locally or look through docker hub.
For example, to fetch a tensorflow image from Docker Hub, first find the image using docker on your own machine (as root).
$ docker search tensorflow NAME DESCRIPTION STARS OFFICIAL AUTOMATED tensorflow/tensorflow Official Docker images for the machine learn… 1828 ..........
$ apptainer pull tensorflow.sif docker://tensorflow/tensorflow Using default tag: latest latest: Pulling from tensorflow/serving f22ccc0b8772: Downloading [===========> ] 6.106MB/26.71MB .........
You now have a Apptainer image tensorflow.sif in your current directory for use with the Apptainer command.
Additional information can be found here.
Working with images
In order to use apptainer on the Beowulf cluster, you need to start an interactive job or submit a batch-job to the available slurm queues.
The below example will illustrate the interactive use of Apptainer in an interactive bash shell.
$ salloc srun --pty $SHELL -l salloc: Granted job allocation 2885955 $ which apptainer /usr/bin/apptainer $ apptainer --version apptainer version 1.1.8-1.sdl9
Shell - Run a shell within a container
$ apptainer shell tensorflow_latest.sif Apptainer> pwd /u/nobody
Commands within a container
Exec - Run a command within a container
$ apptainer exec tensorflow_latest.sif cat /etc/os-release NAME="Ubuntu" VERSION="18.04.5 LTS (Bionic Beaver)" ID=ubuntu
Run - Run the user-defined default command within a container
$ apptainer run tensorflow_latest.sif ________ _______________ ___ __/__________________________________ ____/__ /________ __ __ / _ _ \_ __ \_ ___/ __ \_ ___/_ /_ __ /_ __ \_ | /| / / _ / / __/ / / /(__ )/ /_/ / / _ __/ _ / / /_/ /_ |/ |/ / /_/ \___//_/ /_//____/ \____//_/ /_/ /_/ \____/____/|__/ You are running this container as user with ID xxxxxx and group xxx, which should map to the ID and group for your user on the Docker host. Great! Apptainer containers as Slurm jobs
#!/bin/bash # #*** #*** "#SBATCH" lines must come before any non-blank, non-comment lines *** #*** # # 1 node, 1 CPU per node (total 1 CPU), wall clock time of 30 hours # #SBATCH -N 1 ## Node count #SBATCH --ntasks-per-node=1 ## Processors per node #SBATCH -t 30:00:00 ## Walltime SBATCH --gres=gpu:1 --ntasks-per-node=1 -N 1 # # send mail if the process fails #SBATCH --mail-type=fail # Remember to set your email address here instead of nobody #SBATCH --mail-user=nobody@cs.princeton.edu # apptainer exec tensorflow_latest.sif cat /etc/os-release
References
https://github.com/ArangoGutierrez/Singularity-tutorial
https://hpc.nih.gov/apps/singularity.html