Sandboxing Deep Dive

Namespaces · Capabilities · seccomp-bpf · cgroups v2 · Landlock
bwrap / unshare / nsjail / firejail / podman // there is no "sandbox" syscall, only five primitives // linux 2.4 to 6.x

There Is No Sandbox

Linux has no sandbox syscall, no sandbox object, no sandbox subsystem. It has five independent primitives, each solving a different problem, none of which was designed as a security boundary on its own. Every tool you have heard of is a particular opinion about how to wire them together.

   what each primitive actually controls

   namespaces    what a process can SEE        mounts, pids, net, ipc, uts, users, cgroups, time
   capabilities  which of root's POWERS it holds   ~40 bits, split out of the old all-or-nothing uid 0
   seccomp-bpf   which SYSCALLS it may issue     a bpf program run on every syscall entry
   cgroups v2    how much RESOURCE it consumes    memory, pids, cpu, io
   Landlock      which FILES it may touch        unprivileged, path-based, since 5.13

   bwrap  firejail  nsjail  podman  systemd  gVisor
        all of them = a choice of which of the five, and how tight
Why this framing matters Learn the five and every tool becomes readable. A docker run flag, a firejail profile line, a bwrap argument: each one is turning a knob on one of these. Learn the tools first and you memorize flags without knowing what they protect against, which is how people ship sandboxes with a hole straight through the middle.

The honest strength ladder

   weakest
   1  separate uid + unix perms          what most "sandboxing" actually is
   2  + namespaces, dropped caps         bwrap defaults. good vs accidents
   3  + seccomp                          a parser bug can no longer reach most of the kernel
   4  + Landlock / strict mounts         filesystem reach is explicit, not implied
   5  gVisor                             syscalls served by a userspace kernel
   6  VM (firecracker, qemu)             hardware boundary
   strongest

   analysing a hostile file with a parser  ->  level 3-4 is proportionate
   executing hostile code                  ->  nothing below 6 is honest

Namespaces

Each namespace virtualizes one global kernel resource. A process inside gets its own copy, and cannot see the host's. They are independent: you take only the ones you want.

nsclone flagisolatesnote
mountCLONE_NEWNSthe filesystem treeoldest (2.4). propagation is the subtle part
pidCLONE_NEWPIDprocess IDsfirst process is pid 1 and must reap. it dies, the ns dies
netCLONE_NEWNETinterfaces, routes, netfilter, socketsempty ns has only a down lo
ipcCLONE_NEWIPCsysv ipc, posix mqueues
utsCLONE_NEWUTShostname, domainnamemostly cosmetic
userCLONE_NEWUSERuid/gid maps and capabilitiesthe keystone, see next section
cgroupCLONE_NEWCGROUPthe visible cgroup rootstops escapes via /sys/fs/cgroup paths
timeCLONE_NEWTIMEmonotonic/boottime offsets5.6+. does not virtualize wall clock

See it with no tools at all

# one line. this is the core of every sandbox tool.
  unshare --user --map-root-user --net --pid --fork --mount-proc --uts --ipc bash

# inside:
  ip addr        # only lo. nothing to exfiltrate through
  ps -e          # a handful of processes, not the host's hundreds
  id             # uid 0 ... but see the next section
The three that bite Mount propagation: a bind mount can propagate back to the host if the mount is shared. Hand-rolled unshare frequently gets this wrong and the "sandboxed" process modifies the host. /proc leaks: mounting a fresh /proc without --unshare-pid still shows host processes, so the sandbox reads /proc/<host_pid>/environ for secrets. The two flags are a pair. pid 1 duties: the first process in a pid ns inherits reaping; if it does not, you accumulate zombies.

The Keystone

User namespaces are why unprivileged sandboxing exists at all. Inside one you hold uid 0 with a full capability set, while the kernel maps you to your real unprivileged uid outside. That is what lets a normal user create mount and network namespaces without sudo, and why bwrap needs no setuid bit on a modern kernel.

   inside the namespace          the kernel's view
   uid 0, CAP_SYS_ADMIN,       -- maps to -->   uid 1000, no real privilege
   CAP_NET_ADMIN, ...                            on the host

   root inside is not root outside. capabilities are scoped to the namespace:
   you may mount a tmpfs in YOUR mount ns, not touch the host's.

It is also, historically, a rich source of privilege escalation. Handing unprivileged users reachability into kernel code paths that previously required root turned out to expose a lot of bugs. Distributions responded by restricting it.

Every distribution restricts it, and no two the same way

This is the biggest portability trap in Linux sandboxing. Code that sandboxes cleanly on one machine dies on the next, with an error naming none of the responsible subsystems:

bwrap:   setting up uid map: Permission denied
unshare: write failed /proc/self/uid_map: Operation not permitted
podman:  cannot clone: Operation not permitted

   nothing here names the knob, the LSM, or the policy responsible.
gateseen oncheckmeaning
LSM-mediatedAppArmor distros, 23.10+kernel.apparmor_restrict_unprivileged_usernsallowed only for binaries whose profile grants userns
global toggleDebian-family, hardened kernelskernel.unprivileged_userns_clone0 = no unprivileged userns at all
quotaupstream, every distrouser.max_user_namespaces0 = disabled. RHEL shipped 0 for years
SELinuxFedora, RHELgetenforce + booleansa separate axis, can deny after the others allow
outer sandboxinside a containerthe runtime's seccomp profilenested sandboxing blocked by the layer above

The order matters: a kernel can permit userns by quota and still be denied by an LSM, so reading one knob proves nothing. The only portable probe is to attempt it and read the error.

# the capability test that works everywhere. no sysctl guessing.
unshare --user --map-root-user true && echo "userns available"

Fixes, narrowest scope first

approachscopenote
per-binary LSM profileone executablenarrowest. on AppArmor systems, a few lines granting userns, following the distro's own pattern for tools like mmdebstrap and ch-run
raise the quotasystemuser.max_user_namespaces when it was zeroed
flip the global togglesystemblunt. re-opens exactly what the distro closed on purpose
setuid helperone toolfirejail's approach. trades a userns CVE class for a setuid CVE class
run privileged--often means the payload runs as root inside the sandbox, which is worse than where you started
Be honest about the per-binary grant Granting userns to a sandbox tool is narrower than a global toggle, but only somewhat: that tool exists to create namespaces, so anything able to execute it effectively has userns. On a single-user workstation the difference is small. On a shared host it is not.
Diagnose it in one command ./check probes the capability directly, then, only when it fails, reports which gate above is responsible on your system and which fix applies.

Capabilities

Root's powers split into roughly forty bits, so a process can hold exactly the privilege it needs. A process carries permitted, effective, inheritable, bounding and ambient sets.

capabilitygrantsverdict
CAP_SYS_ADMINmount, pivot_root, many ioctls, and a long tailis root. the junk drawer. granting it means you have not sandboxed anything
CAP_SYS_PTRACEread/write other processes' memorydrop
CAP_DAC_READ_SEARCHbypass file read permission checks entirelydrop
CAP_NET_RAWraw sockets, packet craftingdrop unless the tool genuinely needs it
CAP_SYS_MODULEload kernel modulesnever
Dropping is not enough without no_new_privs A setuid binary executed inside can regain what you dropped. Two things make it stick: clear the bounding set (so caps cannot be re-acquired on exec) and set prctl(PR_SET_NO_NEW_PRIVS, 1), which guarantees no execve ever gains privilege again. no_new_privs is also a precondition: unprivileged seccomp and Landlock both refuse to install without it.

seccomp-bpf

Namespaces decide what a process can see. seccomp decides which kernel APIs it may call at all. It is a classic-BPF program, the same VM lineage as tcpdump filters, run on every syscall entry against a small struct.

   struct seccomp_data -- everything the filter can see
     nr                  syscall number
     arch                AUDIT_ARCH_X86_64, AUDIT_ARCH_I386, ...
     instruction_pointer
     args[6]             the six syscall arguments, as scalars

   verdicts:  ALLOW   ERRNO(fake a failure)   TRAP(SIGSYS)
              KILL_THREAD / KILL_PROCESS   TRACE   USER_NOTIF(5.0+)
seccomp cannot dereference pointers Arguments are visible as scalars only. The filter may not follow a char *path, because userspace could change that memory after the check and before the kernel uses it, a TOCTOU race. So "deny open() of /etc/shadow" is not expressible in a seccomp filter. Path policy has to come from mount namespaces or Landlock.

One nuance the blanket version of this claim misses: a USER_NOTIF supervisor (5.0+) can mediate paths, by copying the argument out of /proc/<tid>/mem once, performing the operation on its own copy, and injecting the resulting fd with SECCOMP_IOCTL_NOTIF_ADDFD. Acting on the supervisor's copy has no TOCTOU. It is the CONTINUE pattern, check-then-let-it-proceed, that stays racy and that the man page says cannot implement a security policy. This is how runc and systemd emulate mount. This single constraint explains most of the design of every sandbox tool.

The classic hand-rolled bug

x86_64 kernels also accept i386 syscalls via int 0x80, where the numbers mean different things. A filter that checks the syscall number without first pinning the architecture can be walked straight around by issuing 32-bit calls. Check arch first, always. libseccomp inserts this for you; hand-written filters must not forget. And pinning the arch is still not sufficient for a denylist on x86_64: x32 syscalls report the same AUDIT_ARCH_X86_64 and are distinguished only by __X32_SYSCALL_BIT (0x40000000) in nr, so a denylist that does not also reject that bit is bypassable. man 2 seccomp says so outright.

/* arch must match, or kill. this is not optional. */
BPF_STMT(BPF_LD  | BPF_W | BPF_ABS, offsetof(struct seccomp_data, arch)),
BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, AUDIT_ARCH_X86_64, 1, 0),
BPF_STMT(BPF_RET | BPF_K, SECCOMP_RET_KILL_PROCESS),

/* only now is it safe to read the syscall number */
BPF_STMT(BPF_LD  | BPF_W | BPF_ABS, offsetof(struct seccomp_data, nr)),
Two policy lessons Prefer ERRNO over KILL. Killing on an unexpected syscall means the next glibc update that calls something new turns a graceful degradation into a crash. Observation-derived profiles are incomplete. Trace a tool on benign input, capture its syscalls, ship the allowlist, and the first error path calls something unlisted and dies. This is exactly why Docker's default profile is a human-maintained allowlist rather than a trace-generated one: it defaults to SCMP_ACT_ERRNO and names the roughly 300 syscalls it permits, leaving about 44 blocked. Curated by people watching kernel CVEs, not derived from a recording of one happy path.

Landlock

An LSM, since 5.13, that lets an unprivileged process restrict its own filesystem access, irreversibly, for itself and everything it spawns. It expresses precisely what seccomp cannot: path policy, without the TOCTOU problem, because the kernel evaluates it at the point of use.

   the model
   1  create a ruleset naming which access rights you take responsibility for
   2  add rules re-permitting specific paths (by open fd, not by string)
   3  landlock_restrict_self()  -- from here it cannot be widened, ever

   rights not named in the ruleset are untouched. a program that never
   declares WRITE cannot restrict writes -- a common first-attempt mistake.

Rights were added across kernel versions, so a program must query the ABI version and mask down to what the running kernel understands. Ask for a right it does not know and the call fails.

Order is load-bearing no_new_privs, then Landlock, then seccomp, then execv. Landlock and unprivileged seccomp both refuse to install without no_new_privs already set, because otherwise a setuid binary could execve the whole sandbox away. See examples/sandbox.c in this repo for the working sequence.

cgroups v2

Resource control rather than an isolation boundary. It will not stop an exploit. It stops a zip bomb, a fork bomb, or a memory hog from taking the host down, which matters because hostile code with no limits will exhaust the machine whether or not it ever escapes.

controllerknobstops
memorymemory.maxdecompression bombs, runaway parsers
pidspids.maxfork bombs
cpucpu.maxa spin loop eating the box
ioio.maxdisk saturation

bwrap has no cgroup support at all. Pair it with systemd, which is the easiest handle on any modern distro:

systemd-run --user --scope -p MemoryMax=512M -p TasksMax=64 -p CPUQuota=50% -- \
  bwrap --unshare-all ... 

The Tools

Same five primitives, different opinions.

toolnsseccompcgroupsunprivilegedcharacter
bwrapallyou supplynoyesthe sharp knife. no policy of its own. Flatpak's engine
firejailallbuilt-in profilespartialsetuidbatteries included, large profile library, setuid is its own attack surface
nsjailallyesyesyesGoogle. many knobs, verbose config, good for jails
systemd-runsomeSystemCallFilter=yesyesalready installed. best cgroup story, weaker fs isolation
podman / dockerallcurated allowlistyesrootless: yesheavy, but that default seccomp profile is genuinely good
gVisor------yesuserspace kernel. much smaller host surface, real perf cost
firecracker / qemu--------a real VM. the only honest answer for executing hostile code
Learn bwrap It has no policy. Every restriction is an explicit flag, which makes a bwrap command line a readable expression of the model rather than a black box. Once you can write one, every other tool becomes "which of these flags did they pick for me".

Three Escapes

None of these involve a kernel bug. All three defeat an otherwise correctly built sandbox, and all three are configuration mistakes people make constantly.

1. D-Bus is a hole straight through everything

Bind the session bus in for desktop-app compatibility and the sandboxed process can talk to host services. The bus name org.freedesktop.Flatpak exposes org.freedesktop.Flatpak.Development.HostCommand, which runs a process on the host for you. That is exactly what flatpak-spawn --host calls (cf. CVE-2019-10063). Do not confuse it with org.freedesktop.portal.Flatpak.Spawn, a different service that by default spawns into another sandbox rather than onto the host. Your namespaces, dropped capabilities and syscall filter are all intact and completely irrelevant, because you handed it an RPC channel to something privileged.

   sandbox --(bind /run/user/1000/bus)--> session bus --> org.freedesktop.Flatpak
                                                        |
                                              .Development.HostCommand() -> runs on the HOST

   fix: do not mount the bus. if unavoidable, front it with xdg-dbus-proxy
        and allowlist specific interfaces.

2. TIOCSTI terminal injection

A process sharing your pty can ioctl(0, TIOCSTI, &c) to push characters into the parent shell's input buffer, which then run as you after the sandbox exits. A full escape via a tty ioctl.

# fix: a fresh session with a detached controlling terminal
bwrap --new-session ...
# newer kernels can also disable it globally, but do not rely on it
sysctl dev.tty.legacy_tiocsti=0

That framing is now out of date in the sandbox author's favour. Since Linux 6.2 TIOCSTI is gated behind CAP_SYS_ADMIN whenever dev.tty.legacy_tiocsti=0, and the build-time CONFIG_LEGACY_TIOCSTI sets that default. Upstream still defaults it on, but major distros ship it off, so on a current mainstream kernel this injection is already dead for unprivileged processes. Keep --new-session anyway: it costs nothing and covers older kernels and anyone who re-enabled the sysctl. (bwrap's own man page cites CVE-2017-5226 for it.)

3. /proc without a pid namespace

Mounting a fresh /proc is not enough by itself. Without --unshare-pid it still shows host processes, so the sandbox reads /proc/<pid>/environ for secrets and /proc/<pid>/fd/* for open files. The two flags are a pair; either alone is a false sense of security.

Test the negative case A sandbox you have not tried to break is a hope, not a control. Every recipe should ship with assertions that the things you forbade actually fail.

Recipes

Untrusted file, parser is the attack surface

The file is never executed. The parser is the exploit. No network, minimal filesystem, no capabilities.

bwrap \
  --unshare-all                  # every ns incl net: cannot phone home
  --new-session                  # no TIOCSTI back into your shell
  --die-with-parent \
  --ro-bind /usr /usr --ro-bind /lib /lib --ro-bind /lib64 /lib64 \
  --proc /proc --dev /dev --tmpfs /tmp --tmpfs /home \
  --ro-bind "$sample" /work/sample   # the ONLY untrusted input, read-only
  --chdir /work --clearenv          # env leaks tokens, paths, LD_PRELOAD
  --setenv PATH /usr/bin:/bin \
  --cap-drop ALL \
  pdfid /work/sample

The inverted case: a network agent

Something that must reach the internet, holding an API key. You cannot stop it talking out, so the goal flips: make sure nothing worth stealing is in reach.

bwrap --unshare-all --share-net \   # isolate everything EXCEPT the network
  --ro-bind /usr /usr --ro-bind /etc/ssl /etc/ssl \   # tls trust store
  --ro-bind /etc/resolv.conf /etc/resolv.conf \
  --bind /home/agent /home/agent \        # the ONLY writable path
  --proc /proc --dev /dev --tmpfs /tmp --cap-drop ALL \
  /home/agent/.local/bin/agent

Run it as a separate uid as well, so even a full sandbox bypass lands somewhere with nothing in it. Two independent layers beat one good one.

Prove it holds

box /dev/null sh -c 'ping -c1 1.1.1.1'   # must fail: no network
box /dev/null sh -c 'cat /etc/shadow'    # must fail: not bound
box /dev/null sh -c 'ps aux | wc -l'     # must be tiny: own pid ns
In this repo check reports what your kernel can actually do and why a sandbox is refusing to start, naming the failing stage rather than guessing from one sysctl. escape-test runs assertions inside a sandbox and reports what leaked, graded against a profile. nsview decides namespace membership by comparing against the kernel's fixed initial-namespace inodes, which beats counting processes. capdecode reads all five capability sets from /proc with no libcap, and calls out the CapEff=0 with a full bounding set shape that reads clean and is not. 01_namespaces_by_hand.sh walks the five primitives with nothing but util-linux, and examples/sandbox.c is self-restriction in C: no_new_privs, Landlock, seccomp, exec, in the order the kernel demands.
// namespaces · userns · caps · seccomp · landlock · cgroups · tools · escapes · recipes //