# Apptainer/Singularity instances
:::{admonition} Overview
:class: note
**Teaching:** 40 min | **Exercises:** 10 min
**Questions**
- How can I keep my container running in the background?
- What are the use cases for instances?
**Objectives**
- Run containers in a detached mode to keep services up.
- Deploy instances via definition files.
:::
As we have studied in previous chapters, commands such as `run` and `shell` allocate Apptainer/Singularity
containers in the foreground, stopping any process running inside the container after logout. This behavior
suits the use case of containers for executing interactive commands in a well-defined environment, but there
are cases when running processes in the background is convenient. For example, when a web application like a
Jupyter notebook is deployed inside a container, it is desired to keep the container up while it waits for connections
from the web browser.
Apptainer provides the concept of _instances_ to deploy services in the background. While Docker is a common
choice of tool for setting services, Apptainer has the advantage of working without requiring any special permissions
(like when you are working in a cluster provided by your university/laboratory).
In this chapter we will learn the basics about their capabilities and some use cases as examples.
## Instances from image files
To start an instance, Apptainer provides the command `instance`. To exemplify, let's pull the AlmaLinux image used
in previous chapters
```bash
apptainer pull docker://almalinux:9
```
The image must be started in the following way:
```bash
apptainer instance start almalinux_9.sif myalma9
```
In this example, the `.sif` is the image downloaded from Docker Hub, and `myalma9` is the name that we have
assigned to the instance. Instead of opening a shell session or executing a command, the container is running in
the background.
Confirm that the instance is running using the `instance list` command
```bash
apptainer instance list
```
```text
INSTANCE NAME PID IP IMAGE
myalma9 3277300 /tmp/myuser/almalinux_9.sif
```
To interact with the instance, the commands `exec` and `shell` are available. The instance must be referred as
`instance://name`.
For example, to open a shell inside the AlmaLinux instance:
```bash
apptainer shell instance://myalma9
```
Remember that exiting the shell instance will not stop the container. For doing so, use `instance stop`:
```bash
apptainer instance stop myalma9
```
You can confirm the instance doesn't exist with `instance list`.
:::{admonition} Instances with bind paths
:class: tip
When starting an instance, the same options for bind directories between the host and the container as running
an interactive session are available. For example, if you want a directory mounted inside the instance, use the
`--bind` option:
```bash
apptainer instance start --bind $HOME/mydata:/data almalinux_9.sif myalma9
```
binding the directory `mydata/` from the host as `/data` inside the instance.
:::
## A web server as an instance
One of the main purposes of the Apptainer instances is deploying services with customized environments. Before moving
to more complex use cases, let's start with a basic example: a web service showing an HTML page with a message.
Let's write a basic `index.html` file as:
```html
My awesome service
Hello world!
If you see this page, my awesome service is up and running.
```
If you are not familiar with HTML take a quick look at the [HTML Tutorial](https://www.w3schools.com/html/), but it is
not mandatory. What really matters is having a minimal webpage that our server will show.
Now, let's prepare a basic web server using [Python http.server](https://docs.python.org/3/library/http.server.html).
Create a definition file, saved as `basicServer.def`, which contains:
```text
Bootstrap: docker
From: ubuntu:24.04
%post
apt-get update -y
apt-get install -y python3
%files
index.html /tmp/index.html
%startscript
cd /tmp
python3 -m http.server 8850
```
If you recall the chapter about [definition files](05-definition-files.md),
this definition file will pull the official Ubuntu image from Docker Hub, and will install Python 3.
In addition, it copies `index.html` to `/tmp` **inside** the container. When the instance starts, commands specified on
`%startscript` are executed. In this example, `http.server` will be executed, serving a page in the port 8850 (you can
use any other port if 8850 is busy with another service).
Let's build an image from the definition. Modern Apptainer versions build images without any special privileges
(with older versions you may need `sudo` or the `--fakeroot` flag):
```bash
apptainer build basicServer.sif basicServer.def
```
Now, let's start an instance named `myWebService` with the image that we just built
```bash
apptainer instance start --no-mount tmp --cleanenv basicServer.sif myWebService
```
Reminder from the previous chapter: with `--no-mount tmp` we are asking Apptainer to NOT bind `/tmp` from the host
to the instance (it is mounted by default), we use instead an isolated `/tmp` inside the instance where index.html has
been copied.
And with `--cleanenv` we clear the environment. It is not always necessary but it prevents interference
from the host environment (see the [Building Containers episode](04-building-containers.md)).
You can confirm in the terminal that the web service is up using `curl` as
```bash
curl http://localhost:8850
```
```text
My awesome service
Hello world!
If you see this page, my awesome service is up and running.
```
If you are executing Apptainer locally, try to open http://localhost:8850.
:::{admonition} SSH tunneling
:class: tip
If you are deploying a service in a cluster of your institution (such as LXPLUS at CERN) it is likely that you need
SSH tunneling for opening pages served by your service with a web browser. A basic Local Port Forwarding can be configured as:
```bash
ssh -L [local_hostname:]:localhost: myuser@
```
where `` is the port used by your service running on ``, and `` is the address of your institutional resources
(`echo "$(whoami)@$(hostname)"` on the server will print your user and host).
Finally `localhost:` is how you will access the service locally, e.g. what you type in the laptop browser.
For example, for connecting to LXPLUS forwarding the port 8850:
```bash
ssh -L 8850:localhost:8850 myuser@lxplus.cern.ch
```
Then you can open http://localhost:8850 in your machine!
Port numbers less than 1024 are privileged ports and can be used only by root.
Please consult the allowed ports and rules with your institution.
:::
Remember to stop the instance once you are done.
## Serving a Jupyter notebook with custom environment
As an example of the capabilities of instances as services, let's extend our definition file to deploy a
Jupyter notebook server with a customized environment.
[Jupyter Notebook](https://jupyter.org/) is a web-based interactive computing platform. The notebook combines live code, equations,
narrative text, and visualizations.
What if we provide a Jupyter notebook ready to use ROOT? If you remember our example from the
[definition files chapter](05-definition-files.md),
at this point it must be almost straightforward:
```text
Bootstrap: docker
From: ubuntu:24.04
%post
apt-get update -y
apt-get install -y python3
apt-get install -y python3-pip
apt-get install -y python3-notebook
apt-get install wget -y
export DEBIAN_FRONTEND=noninteractive
apt-get install dpkg-dev cmake g++ gcc binutils libx11-dev libxpm-dev \
libxft-dev libxext-dev libssl-dev libgsl-dev libtiff-dev libtbb-dev -y
cd /opt
wget https://root.cern/download/root_v6.38.04.Linux-ubuntu24.04-x86_64-gcc13.3.tar.gz
tar -xzvf root_v6.38.04.Linux-ubuntu24.04-x86_64-gcc13.3.tar.gz
%environment
export PATH=/opt/root/bin:$PATH
export LD_LIBRARY_PATH=/opt/root/lib:$LD_LIBRARY_PATH
export PYTHONPATH=/opt/root/lib
%startscript
jupyter notebook --port 8850
```
Save the definition file as `jupyterWithROOT.def`, and let's build an image called `jupyterWithROOT.sif`
```bash
apptainer build jupyterWithROOT.sif jupyterWithROOT.def
```
Now, start an instance named `mynotebook` with our brand-new image.
Consider using `--cleanenv` if needed.
```bash
apptainer instance start jupyterWithROOT.sif mynotebook
```
and confirm that the instance is up
```bash
apptainer instance list
```
```text
INSTANCE NAME PID IP IMAGE
mynotebook 10720 /home/myuser/jupyterWithROOT.sif
```
If you go to http://localhost:8850 (with SSH tunneling if needed), you will find out that for security reasons the
Jupyter webapp will ask for an access token. Fortunately, you can get the token listing the URL of active servers using
the `jupyter notebook list` command. To execute the command inside the instance, use `apptainer exec`:
```bash
apptainer exec instance://mynotebook jupyter notebook list
```
```text
Currently running servers:
http://localhost:8850/?token=12asldc9b2084f9b664b39a6246022312bc9c605b :: /home/myuser
```
:::{admonition} Notebook starting on a different port!
:class: tip
If the chosen port for the Notebook (8850, set in the definition file) is not available,
the notebook will not error out, but will start and use the first available port after that.
E.g. if you did not terminate the web server from the previous example, the notebook will start
on a different port; the `jupyter notebook list` command shown above will show you the correct port.
:::
Open the URL with the token (from http to the first space), and you will be able to see the Jupyter interface. Try to open a new notebook and write in
a cell to confirm that ROOT is available:
```python
import ROOT
# Now you can work with PyROOT, creating a histogram for example
h = ROOT.TH1F("myHistogram", "myTitle", 50, -10, 10)
h.FillRandom("gaus", 10000)
c = ROOT.TCanvas("myCanvasName","The Canvas Title",800,600)
h.Draw()
c.Draw()
```
The bottom line: with any Jupyter notebook that you write, you can provide an Apptainer image that will
set the environment required to execute the cells. It doesn't matter whether you or someone else comes back in one, five,
or ten years: your code will work independently of the software available on your computer as long as Apptainer/Singularity
is available!
::::{admonition} A Jupyter notebook with Uproot available
:class: important
Can you set up a Jupyter notebook server with [Uproot](https://uproot.readthedocs.io/en/latest/index.html) available in Apptainer?
Hint: Uproot can be installed using `pip`. And use the option `--break-system-packages`.
New Python versions complain when installing packages without a virtual environment. You need that option to force the install.
:::{admonition} Solution
:class: dropdown
```text
Bootstrap: docker
From: ubuntu:24.04
%post
apt-get update -y
apt-get install -y python3
apt-get install -y python3-pip
apt-get install -y python3-notebook
pip install --break-system-packages uproot
%startscript
jupyter notebook --port 8850
```
Confirm that Uproot is available opening a notebook and executing in a cell
```python
import uproot
print(uproot.__doc__)
```
:::
::::
:::{admonition} Key Points
:class: note
- Instances allow you to set up services via Apptainer images or definition files.
- Code provided in Jupyter notebooks can be accompanied by an Apptainer/Singularity image with the environment needed for its execution, ensuring the reproducibility of the results.
:::