Containers from definition files

Containers from definition files#

Overview

Teaching: 20 min | Exercises: 20 min

Questions

  • How to easily build and deploy containers from a single definition file?

Objectives

  • Create a container from a definition file.

As shown in the previous chapter, building containers with an interactive session may take several steps, and it can become as complicated as the required setup. An Apptainer definition file provides an easy way to build and deploy containers.

Hello World Apptainer#

The following recipe shows how to build a hello-world container, and run the container on your local computer.

  • Step 1: Open a text editor (e.g., nano, vim, or gedit in a graphical environment)

    nano hello-world.def

  • Step 2: Include the following script in the hello-world.def file to define the environment

    Bootstrap: docker
From: ubuntu:24.04

%runscript
  echo "Hello World"
# Print Hello world when the image is loaded

    In the above script, the first line - Bootstrap: docker indicates that Apptainer will use the Docker protocol to retrieve the base OS to start the image. The From: ubuntu:24.04 is given to Apptainer to start from a specific image/OS in Docker Hub. Any content within the %runscript will be written to a file that is executed when one runs the apptainer image. The echo "Hello World" command will print the Hello World on the terminal. Finally the # hash is used to include comments within the definition file.

  • Step 3: Build the image

    apptainer build hello-world.sif hello-world.def

    The hello-world.sif file specifies the name of the output file that is built when using the apptainer build command.

  • Step 4: Run the image

    ./hello-world.sif

Deleting Apptainer image#

To delete the hello-world Apptainer image, simply delete the hello-world.sif file.

apptainer delete

Note that there is also an apptainer delete command, used to delete an image from a remote library. To learn more about using remote endpoints and pulling and pushing images from or to libraries, read Remote Endpoints and Library API Registries.

Example of a more elaborated definition file#

Let’s look at the structure of the definition file with another example. Let’s prepare a container from an official Ubuntu image, but this time we will install ROOT with RooFit and Python integration.

Following the ROOT instructions to download a pre-compiled binary distribution, the definition file will look like

Bootstrap: docker
From: ubuntu:24.04

# NOTE: This section is only if building the container in the JupyterHub at CompHEP 2026
%setup
    mkdir -p ${APPTAINER_ROOTFS}/cvmfs
    mkdir -p ${APPTAINER_ROOTFS}/direct/u0b
    mkdir -p ${APPTAINER_ROOTFS}/u0b/software

%post
    apt-get update -y
    export DEBIAN_FRONTEND=noninteractive
    apt-get install wget -y
    apt-get install binutils cmake dpkg-dev g++ gcc libssl-dev git libx11-dev \
        libxext-dev libxft-dev libxpm-dev python3 libtbb-dev libvdt-dev libgif-dev \
        libgsl-dev -y
    cd /opt
    wget https://root.cern/download/root_v6.38.04.Linux-ubuntu24.04-x86_64-gcc13.3.tar.gz
    tar -xzvf root_v6.38.04.Linux-ubuntu24.04-x86_64-gcc13.3.tar.gz

%environment
    export PATH=/opt/root/bin:$PATH
    export LD_LIBRARY_PATH=/opt/root/lib:$LD_LIBRARY_PATH
    export PYTHONPATH=/opt/root/lib

%runscript
    python3 /opt/root/tutorials/roofit/roofit/rf101_basics.py

%labels
    Author HEPTraining
    Version v0.0.1

%help
    Example container running the RooFit tutorial and producing the rf101_basics.png image.
    The container provides ROOT with RooFit and Python integration running on Ubuntu.

What is export DEBIAN_FRONTEND=noninteractive for?

Some Debian/Ubuntu packages pause during installation to ask configuration questions (for example, tzdata asks for your geographic region and time zone). During apptainer build there is no terminal to type answers into, so a prompt like this would hang the build. Setting DEBIAN_FRONTEND=noninteractive tells apt-get to skip all prompts and accept the default answers, letting the build run unattended. It only affects the commands that follow it inside %post; it does not change the environment of the final container.

Let’s take a look at the definition file:

  • The first two lines define the base image. In this case, the image ubuntu:24.04 from Docker Hub is used.

  • %post are lines to execute inside the container after the OS has been set. In this example, we are listing the steps that we would follow to install ROOT with a precompiled binary in an interactive session. Notice that the binary used corresponds with the Ubuntu version defined at the second line.

  • %environment is used to define environment variables available inside the container. Here we are setting the env variables required to execute ROOT and PyROOT.

  • Apptainer containers can be executable. %runscript define the actions to take when the container is executed. To illustrate the functionality, we will just run rf101_basics.py from the RooFit tutorial.

  • %labels add custom metadata to the container.

  • %help is the container documentation: what it is and how to use it. It can be displayed using apptainer run-help.

Save this definition file as rootInUbuntu.def. To build the container, just provide the definition file as argument. Modern Apptainer versions build images without any special privileges; with older versions you may need sudo or the --fakeroot option:

apptainer build rootInUbuntu.sif rootInUbuntu.def

Then, an interactive shell inside the container can be initialized with apptainer shell, or a command executed with apptainer exec. A third option is to execute the actions defined inside %runscript simply by calling the container as an executable

./rootInUbuntu.sif

...
 PARAMETER  CORRELATION COEFFICIENTS
       NO.  GLOBAL      1      2
        1  0.02723   1.000  0.027
        2  0.02723   0.027  1.000
[#1] INFO:Minimization -- RooMinimizer::optimizeConst: deactivating const optimization
RooRealVar::mean = 1.01746 +/- 0.0300144  L(-10 - 10)
RooRealVar::sigma = 2.9787 +/- 0.0219217  L(0.1 - 10)
Info in <TCanvas::Print>: png file rf101_basics.png has been created

You will find the output file rf101_basics.png in the location where the container was executed. The exact output depends on the ROOT version used in the container. If you don’t have a DISPLAY set up, ROOT may complain. Ignore the error messages; the image will be created anyway.

Here we have covered the basics with a few examples focused on HEP software. Check the Apptainer docs to see all the available options and more details related to the container creation.

A few best practices for your containers to make them more usable, portable, and secure:

  1. Always install packages, programs, data, and files into operating system locations (e.g. not /home, /tmp , or any other directories that might get commonly binded on).

  2. Document your container. If your runscript doesn’t supply help, write a %help or %apphelp section. A good container tells the user how to interact with it.

  3. If you require any special environment variables to be defined, add them to the %environment and %appenv sections of the build recipe.

  4. Files should always be owned by a system account (UID less than 500).

  5. Ensure that sensitive files like /etc/passwd, /etc/group, and /etc/shadow do not contain secrets.

  6. Build production containers from a definition file instead of a sandbox that has been manually changed. This ensures the greatest possibility of reproducibility and mitigates the “black box” effect.

Deploying your containers

Keep in mind that, while building a container may be time consuming, the execution can be immediate and anywhere your image is available. Once your container is built with the requirements of your analysis, you can deploy it in a large cluster and execute it as far as Apptainer is available on the site.

Libraries like Sylabs Cloud Library ease the distribution of images. Your institution (e.g. Fermilab or CERN) may provide a Harbor registry. GitHub has a Container Registry that Apptainer can access via the ORAS API. Organizations like OSG provide instructions to use available images and distribute custom images via CVMFS.

Be smart, and this will open endless possibilities in your workflow.

Write a definition file to build a container with Pythia8 available in Python

Following the example of the first section in which a container is built with an interactive session (see the previous episode), write a definition file to deploy a container with Pythia8 available.

Take a look at /opt/pythia/pythia8310/examples/main01.py and define the %runscript to execute it using python3.

(Tip: notice that main01.py requires Makefile.inc).

Solution

Bootstrap: docker
From: almalinux:9

%post
    dnf -y groupinstall 'Development Tools'
    dnf -y install python3-devel
    mkdir /opt/pythia && cd /opt/pythia
    curl -o pythia8310.tgz https://pythia.org/download/pythia83/pythia8310.tgz
    tar xvfz pythia8310.tgz
    cd pythia8310
    ./configure --with-python-include=/usr/include/python3.9
    make

%environment
    export PYTHONPATH=/opt/pythia/pythia8310/lib:$PYTHONPATH
    export LD_LIBRARY_PATH=/opt/pythia/pythia8310/lib:$LD_LIBRARY_PATH

%runscript
    cp /opt/pythia/pythia8310/Makefile.inc .
    python3 /opt/pythia/pythia8310/examples/main01.py

%labels
    Author HEPTraining
    Version v0.0.2

%help
    Container providing Pythia 8.310. Execute the container to run an example.
    Open it in a shell to use the Pythia installation with Python 3.9

Save this definition file as myPythia8.def and build your container executing

apptainer build pythiaInAlma9.sif myPythia8.def

And finally, execute the container to run main01.py

./pythiaInAlma9.sif

This solution is building Pythia from scratch and may take several minutes to build the container. In the previous episode we saw that binary packages of Pythia are available in EPEL. Use them to build a similar container much faster.

Key Points

  • An Apptainer definition file provides an easy way to build and deploy containers.