The per-snap mount namespace
sandbox internals for snapd developers for administrators
snap-confine builds one mount namespace per snap on the first run with the base snap as the root and keeps it as a file under /run/snapd/ns. Later runs join it. A namespace that no longer matches the base is rebuilt once no process of the snap is left in it, and a refresh or removal discards it.
On the map: sandbox internals district
01 One view of the filesystem per snap
Each strictly confined snap gets one mount namespace, with its base snap as the root. Mounts on the host propagate into it, mounts made inside do not propagate out, and removable media is shared both ways. It is built on the first run and shared by every app and hook of the snap. Classic snaps do not get one.
You list a system directory in a terminal, then open the same directory in photo editor's file dialog, and the two don't agree. Nothing is broken. Photo editor has its own view of the filesystem. Its root is the base snap, so it finds the same libraries on every distribution. Your home is there as well. When something is mounted on the laptop, it shows up inside, but a mount made inside stays inside. removable-media is the exception: it's shared both ways. The view is built the first time photo editor runs, then kept, and every app and hook of the snap shares it. Classic snaps don't get one.
02 The first run builds it for app developers for Ubuntu Core developers for snapd developers for administrators
Lock, helper, unshare(CLONE_NEWNS), a tmpfs root carrying the base's top-level entries, bind mounts from the host, pivot_root with the old root at /var/lib/snapd/hostfs, a private /tmp, a new devpts, then snap-update-ns --from-snap-confine.
It's the first time photo editor runs since boot, so there's nothing to join yet. snap-confine builds the view. It takes photo editor's lock, so two starts can't build at once, and forks a helper that stays outside. Then it unshares. An empty in-memory filesystem becomes the new root, and core24's top-level directories are bound onto it. The host's devices, configuration, homes, proc, sys and run follow, with media shared both ways. pivot_root swaps the roots and leaves the old one at /var/lib/snapd/hostfs. Last come a private tmp, a fresh devpts, and one run of snap-update-ns for interface mounts and layouts.
03 Kept as a file for Ubuntu Core developers for snapd developers for administrators
<instance>.mnt is a bind mount of the namespace's nsfs file, made by the helper process. snap.<instance>.info records base-snap-name and snap.<instance>.fstab is the current mount profile. snap debug mount-namespace <snap> reports it and --shell enters it.
A namespace normally vanishes when its last process exits. So how does photo editor's survive you closing the window? snap-confine pins it. Every namespace has a file under proc, and bind-mounting that file somewhere keeps the namespace alive. The mount has to be made from outside, where the /run/snapd/ns directory is visible, and that's the helper's job. The result is photo-editor.mnt. Next to it sit an info file with the base's name, and the current mount profile. Snap debug mount-namespace tells you whether one exists, and with the shell option it gives you a shell that sees what photo editor sees.
04 Later runs join for Ubuntu Core developers for snapd developers
snap-confine opens <instance>.mnt and checks it is nsfs. A child joins and votes SC_DISCARD_NO, SC_DISCARD_SHOULD or SC_DISCARD_MUST, then the parent calls setns. Later changes are applied by snapd through snap-update-ns.
You open a second photo editor window. snap-confine finds the file from last time, and the question becomes: is what's in there still good? It opens photo-editor.mnt and checks it's really a namespace. Then it forks a child to go in and look, because the kernel misbehaves if one process calls setns twice. The child votes. It fits, so reuse it. It's stale, so discard it if nobody's inside. Or the snap moved to another base: discard it regardless. On reuse, snap-confine joins with a single setns. Nothing is mounted on this path, which is why a later connection needs snap-update-ns.
05 Stale, but someone is still inside
A view whose root is not the current revision of the base is stale. It is reused while any process of the snap runs, and rebuilt on the first start after the last one exits. The revision numbers are invented for this deck.
core24 just refreshed, and photo editor is still open on your screen. Which core24 does a new photo editor window see? The old one. Photo editor's view was built on revision 1100, and the window that's still open lives in it. snap-confine notices the view is stale, sees that photo editor is still running, and lets the new window join. Two halves of one app looking at two different bases would be worse. Once every photo editor process exits, the next start throws the old view away and builds a fresh one on 1150. So if a base refresh seems to change nothing, check what's still running.
06 A second layer for each user for app developers for Ubuntu Core developers for snapd developers for administrators
For a non-root user snap-confine unshares a second time and runs snap-update-ns --user-mounts if snap.<instance>.user-fstab exists. The per-user namespace is not preserved. The entry shown is the one the desktop interface adds.
Everything so far is shared by every user of the laptop. But some mounts belong to one person. The desktop interface, for example, wants the document portal's folder under your own runtime directory. So when anyone but root starts photo editor, snap-confine unshares once more, on top of the shared view. If the snap has a user mount profile, snap-update-ns applies it, with $XDG_RUNTIME_DIR filled in as /run/user/1000. If there's no profile, the step returns at once. This layer isn't kept. Every start by a user builds it again on top of the shared one.
07 Who throws it away for app developers for Ubuntu Core developers for snapd developers for administrators
snap-discard-ns <instance> unmounts and removes <instance>.mnt and the .fstab and .info files. snapd runs it on a refresh that waited for the apps to close (not with --ignore-running, and not when a service uses refresh-mode: endure), on removal and when an update of the namespace fails; snap-confine runs it for a stale, unoccupied namespace.
You refresh photo editor to 2.5. Its view was built for revision 42, so what happens to it? Snapd discards it while the snap is locked and no app is running. The tool is snap-discard-ns. It unmounts the namespace file and deletes it, with the info file and the current profile, and the next start builds a fresh view. Removing the snap does the same. snap-confine calls it for a stale view nobody's using, and you can run it by hand. One exception: a service with refresh-mode: endure keeps running across a refresh, so its view stays. Don't count on a mount surviving a refresh.
08 Where to go from here
Roads: snap-update-ns, the system key, hotplug slots, confinement, snap debug, services, parallel installs and the security backends.
So that's the view: built once, kept as a file, joined by everything that follows. When a connection or a layout changes, snap-update-ns edits the kept view in place; that's next door. The system key answers a different question: do the profiles on disk still fit this system? Hotplug covers slots that appear when you plug a device in. Confinement shows the rest of what snap-confine sets up, and snap debug lists the other tools for looking inside. Services explains services that keep running across a refresh.
Read more
for administrators for snapd developers for Ubuntu Core developers
Files
<instance> is the snap instance name, such as photo-editor or photo-editor_beta.
| Path | Written by | Content |
|---|---|---|
/run/snapd/ns/ | snap-confine | a directory, bind-mounted over itself and made private (MS_PRIVATE) |
/run/snapd/ns/<instance>.mnt | the snap-confine helper process | a bind mount of the nsfs file of the snap's mount namespace; mode 0600 |
/run/snapd/ns/<instance>.<uid>.mnt | nothing at present | the per-user namespace file; the code to capture it exists but is not called |
/run/snapd/ns/snap.<instance>.info | snap-confine | base-snap-name=<name>, and managed-ca-certs-generation=<id> when managed CA certificates are mounted |
/run/snapd/ns/snap.<instance>.fstab | snap-confine, then snap-update-ns | the current mount profile; its first line, written by snap-confine, is tmpfs / tmpfs x-snapd.origin=rootfs 0 0 |
/run/snapd/ns/snap.<instance>.log and snap.<instance>.user.<uid>.log | you, by hand | if present, snap-update-ns appends debug logs there. Nothing in snapd creates or removes them |
/run/snapd/lock/<instance>.lock and /run/snapd/lock/.lock | snap-confine | the per-snap and the global flock files |
/var/lib/snapd/mount/snap.<instance>.fstab | snapd's mount backend | the desired mount profile |
/var/lib/snapd/mount/snap.<instance>.user-fstab | snapd's mount backend | the desired per-user mount profile |
/tmp/snap-private-tmp/snap.<instance>/tmp | snap-confine | becomes /tmp inside the namespace |
/tmp/snap-private-tmp/snap.rootfs_XXXXXX | snap-confine | the scratch directory for the new root; removed after pivot_root |
What snap-confine bind-mounts from the host
- Always:
/dev,/etc,/home,/root,/proc,/sys,/tmp,/var/snap,/var/lib/snapd,/var/tmp,/run,/usr/src,/var/log. - When present:
/lib/modules,/lib/firmware,/mnt,/var/lib/extrausers. - Shared both ways:
/media(on systems built with a merged/usr,/run/mediawith/mediaas the alternative) and/run/netns. - Home directories configured outside
/home, added on every build of the namespace. - The old root stays reachable at
/var/lib/snapd/hostfs. Itssys,devandprocare detached, and on Ubuntu Core so is/var/lib/snapd/hostfs/writable.
The other top-level directories come from the base snap: directories are bind-mounted onto the new tmpfs root, symbolic links are recreated and regular files are bind-mounted. /snap and the directories in the lists above are skipped.
When a namespace is stale
snap-confine treats a kept namespace as stale when any of these holds:
- the current revision of the base snap is not what is mounted at
/,/usror under/usr/in the namespace. This is what a base refresh causes; - a configured home directory is not a mount point in the namespace;
- the managed CA certificate generation differs from the one in the
.infofile, or that mount is missing. The mount is only made on Ubuntu Core-like distributions whose base has/etc/ssl/certs.
A stale namespace that a process of the snap still uses is reused, with the debug message "preserved mount namespace is stale but occupied, reusing". Use is read from the freezer cgroup on cgroup v1 and from the tracking cgroups on cgroup v2.
A change of base, for example from core22 to core24, is read from base-snap-name in the .info file. It discards the namespace even when it is in use. When the file or the key is absent no change is detected.
Stale detection is off in legacy mode (Ubuntu Core 16 with the base core); the base change check is not.
Who discards
| Caller | When | Message |
|---|---|---|
| snap-confine | stale and not in use, or the base name changed | |
the unlink-current-snap task | on refresh, while the snap lock is held and no app runs; skipped when a service has refresh-mode: endure; also skipped with snap refresh --ignore-running and for snap types that refresh without waiting for apps to close | not discarding snap mount namespace: service "<name>" uses endure refresh-mode in the task log |
the discard-snap task | when the last revision is removed | cannot discard snap namespace "<snap>", will retry in 3 mins: ... |
undo of link-snap | when a first install fails | |
| a snapd downgrade below 2.57 | for every snap | Downgrading snapd to version "<version>", discarding preserved namespaces |
| the mount backend, on removal of a snap's profiles | after removal | |
| the mount backend, after a failed update | when snap-update-ns fails, unless a daemon uses refresh-mode: endure | cannot update mount namespace of snap "<snap>", and cannot discard it because it contains an enduring daemon: ... |
| the mount backend, delayed | for a snap that consumes an updated slot provider, if none of its apps runs | |
snap debug mount-namespace --discard <snap> | by hand | cannot discard mount namespace of snap "<snap>": ... |
snapd runs the tool only if /run/snapd/ns/<instance>.mnt exists. Its own error is cannot discard preserved mount namespace of snap "<snap>": ....
Classic snaps and parallel instances
Classic snaps do not get a per-snap mount namespace. The snap documentation says the same. There is one exception in the code: with the parallel-instances feature on and an instance key, snap-confine unshares a mount namespace for a classic snap and bind-mounts the <snap>_<key> directories over <snap>. That namespace is not kept. snapd's code calls these parallel instances; the documentation calls them parallel installs.
Commands
| Command | Effect |
|---|---|
snap debug mount-namespace <snap> | prints mount namespace of snap "<snap>" bound to /run/snapd/ns/<snap>.mnt, use --shell to enter or no mount namespace of snap "<snap>" found |
snap debug mount-namespace --shell <snap> [-- <command>] | runs nsenter -m on the file with /bin/bash or the given command. The help says the command may need root |
snap debug mount-namespace --discard <snap> | runs snap-discard-ns. --shell and --discard cannot be used together |
snap-discard-ns [--snap-already-locked or --from-snap-confine] <instance> | an internal tool; needs CAP_SYS_ADMIN, CAP_DAC_OVERRIDE and CAP_CHOWN. With either flag it checks that the snap lock is already held; without, it takes the lock. It unmounts and removes <instance>.mnt, <instance>.*.mnt, snap.<instance>.fstab, snap.<instance>.*.user-fstab and snap.<instance>.info, and exits 0 if /run/snapd/ns does not exist |
for snapd developers for administrators
Messages
From snap-confine, each ending the run:
cannot create directory %s,cannot self-bind mount %scannot open preserved mount namespace %s,cannot join preserved mount namespace %ssupport process for mount namespace inspection exited abnormallycannot preserve mount namespace of process %d as %s,helper process exited abnormallycannot unshare the mount namespacecannot perform operation: pivot_root %s %s%s failed with code %i, when snap-update-ns or snap-discard-ns failscannot find mount entry for snap %s revision %ssanity timeout expired, when a lock is not acquired within 30 secondssnap is currently being removed
From snap-discard-ns: missing capability %s.
From snapd: cannot discard preserved mount namespace of snap %q: %s.
for snapd developers
Details for snapd developers
- snap-confine first moves into the mount namespace of PID 1 if it is not there already, so that
/run/snapd/nsshows the kept namespaces even when it is started from inside a snap. - Legacy mode is
SC_DISTRO_CORE16with the basecore. It does not pivot into a base, bind-mounts/recursively instead of building a tmpfs root, and skips stale detection. - On kernel 6.18 only, after
unshare(CLONE_NEWNS)snap-confine checks with theNS_GET_IDioctl that the new namespace id is greater than the helper's, and unshares again on other CPUs until it is. pivot_rootis used rather thanchrootso that AppArmor sees/as the root in every case.- The child that inspects a kept namespace reports through an eventfd: 1 for
SC_DISCARD_NO, 2 forSC_DISCARD_SHOULD, 3 forSC_DISCARD_MUST. sc_preserve_populated_per_user_mount_ns()exists but snap-confine does not call it, so<instance>.<uid>.mntis not created.- The mount backend advertises the
per-snap-user-profilessandbox feature. - The README next to the code (
cmd/snap-confine/README.mount_namespace) describes older paths for the scratch directory and the private/tmp; the code is as described here. - Spread tests:
tests/main/mount-ns(mountinfo tables for core16 and core18 snaps, the per-user namespace and a classic snap),tests/main/snap-discard-ns,tests/main/stale-base-snap(a running process keeps the old base visible to new ones),tests/main/user-mounts(thedesktopuser-fstab entry).
Further reading
- Security policies: the launch steps as the snap documentation lists them, including the private mount namespace.
- Security and sandboxing: the same steps for Ubuntu Core.
- Classic confinement: what is skipped for classic snaps. It says classic snaps have no mount namespace; see the section on parallel instances above for the one exception in the code.
- Parallel installs: the path mapping done inside the namespace for an instance.
No published page explains how the namespace is kept between runs or what snap-discard-ns does.
Roads from here
INTERFACES AND CONFINEMENT
- Confinement — the rest of the sandbox that snap-confine sets up
- How interfaces are implemented — the security backends including the mount backend
RUNNING SNAPS
- Services — services and what refresh-mode endure means for them
TOOLS
- snap debug: tools for looking inside — the other snap debug commands for looking inside
ADMINISTRATION · managing a system of snaps
- Parallel installs — the instance mapping done inside this namespace
SANDBOX INTERNALS · the mount namespace
- snap-update-ns and the mount profiles — how the mounts inside a kept namespace are changed later
- The system key — how snapd knows the profiles on disk still fit this system
- Hotplug slots — slots that appear when a device is plugged in