Profiles for security-sensitive programs

precise rules, every operation for application developers for administrators for security researchers

Privileged daemons and setuid helpers and network-facing services get profiles written by someone who knows every operation the program performs. Capabilities are listed one by one with reasons and paths are exact and every program started gets its own or a child profile. Shown on the shipped ping and dnsmasq profiles with the places where shipped profiles fall short.

On the map: security-sensitive programs

▶ Watch this deck — 10 scenes, ~8 min

Worth seeing first

Part of journeys

01 A profile for a program that holds privileges

SECURITY-SENSITIVE PROGRAMS unix_chkpwd checks your password # To write records to the kernel auditing log. capability audit_write, # To read shadow with 000 permissions. capability dac_read_search, network netlink raw, @{exec_path} mr, /etc/shadow r, plus base and nameservice: see the next slide a reason per privilege /etc/shadow read rules this profile does not have /etc/shadow w, /etc/** r, capability dac_override, the author knew each operation: which call, which file, which capability, and why
The password checker unix_chkpwd and the body of its shipped profile: two capability rules, each under a comment that gives the reason, and the single rule /etc/shadow r. A note says the profile also includes base and nameservice. Three rules the profile does not have are shown as refused: write to /etc/shadow, read of all of /etc, and capability dac_override.

unix-chkpwd: capability audit_write, and capability dac_read_search, each with a reason, and /etc/shadow r,. A precise profile lists individual capabilities, exact paths and named exec targets.

unix_chkpwd is the little helper that checks your password. To do that it reads the shadow file, so if it's ever taken over, the attacker is reading password hashes. Now look at how its shipped profile is written. Two capabilities, each with a comment that says what it's for. One exact rule: /etc/shadow, read. Not write, and not /etc as a whole. You can only write a profile like this if you know every operation the program performs. This deck shows how such profiles are built, how their authors find out what the program does, and where shipped ones still fall short.

Details and sources

Details

  • The profile is unix-chkpwd, attached to /{,usr/}{,s}bin/unix_chkpwd (unix-chkpwd:14). Its two capabilities each sit under a comment that gives the reason: capability audit_write, "To write records to the kernel auditing log." and capability dac_read_search, "To read shadow with 000 permissions." (unix-chkpwd:18-21).
  • The only rule for the password database is /etc/shadow r, (unix-chkpwd:27). The only network rule of its own is network netlink raw, (unix-chkpwd:23).
  • The three red chips are rules the profile does not contain. They are not quoted from a log.
  • It still includes two abstractions, abstractions/base and abstractions/nameservice (unix-chkpwd:15-16), which add their own rules; the next scene shows what nameservice brings.
  • owner /dev/pts/[0-9]* rw, and owner /dev/tty[0-9]* rw, stand under the comment "file_inherit" (unix-chkpwd:37-39): terminals the helper inherits already open.
  • The profile line carries flags=(attach_disconnected.path=/att/unix-chkpwd/), and five rules name paths below that prefix, for example /att/unix-chkpwd/run/systemd/journal/dev-log w, (unix-chkpwd:41-46). See The other profile flags.
  • By the numbers: 22 rule lines, 2 abstractions, 2 capabilities, no **, 2 owner rules.
  • Nothing on this slide was observed on a running kernel; kernel behaviour is read from the kernel source, profiles from AppArmor 836052d21 (6.0.0~alpha2).

02 ping next to curl: what precise means

TWO SHIPPED PROFILES ping include <abstractions/base> include <abstractions/consoles> include <abstractions/nameservice> capability net_raw, capability setuid, network inet raw, network inet6 raw, @{exec_path} mixr, /etc/modules.conf r, @{PROC}/sys/net/ipv6/conf/all/disable_ipv6 r, curl 6 includes owner file rw @{HOME}/**, network inet stream, ... (6 rules) file rw @{run}/snapd.socket, ping curl capability rules 2 0 rules with ** 0 1 capability net_bind_service, network inet stream, network inet dgram, (and inet6) network netlink raw, also in ping after expansion from abstractions/nameservice newer precise profiles include nameservice-strict instead (iotop-c, hostname, nslookup)
The complete body of the shipped ping profile on the left: three includes, two capabilities, two raw socket rules and three file rules. A summary of the curl profile on the right with its rule for the whole home directory. A tally: ping has two capabilities and no rule with two stars, curl has no capability and one rule with two stars. Below, the rules ping also gets through abstractions/nameservice: capability net_bind_service, inet stream and dgram sockets and netlink raw.

ping: capability net_raw, capability setuid, network inet raw, network inet6 raw, and two exact paths. Through abstractions/nameservice it also gets capability net_bind_service, inet stream and dgram sockets and network netlink raw.

Here are the two shipped profiles from the general-purpose deck, with ping in front this time. On the left is ping's whole profile: two capabilities, two raw socket rules, three file rules, and you could explain every one. Curl, on the right, has a rule for your entire home directory. That's the difference in style. But be honest about the left side too. Ping includes the nameservice abstraction, the same one curl uses, so after expansion it also has ordinary TCP and UDP sockets and a third capability. Precise is a discipline, not something a profile gets by being short. Newer profiles take the nameservice-strict abstraction instead.

Details and sources

Details

  • ping's whole body is on the slide: bin.ping:15-27. curl is summarised from curl:14-48: six abstraction includes, owner file rw @{HOME}/**, (curl:33), six network rules (curl:36-41) and file rw @{run}/snapd.socket, (curl:45).
  • What abstractions/nameservice adds to every profile that includes it: network inet stream, network inet6 stream, network inet dgram, network inet6 dgram, and network netlink raw, (nameservice:92-99), and through abstractions/nis capability net_bind_service, (nis:16). The parser built from the AppArmor tree, run on bin.ping, lists them next to ping's own two capabilities.
  • Profiles that include abstractions/nameservice-strict instead: iotop-c:8, hostname:21, nslookup:20, mbsync:18.
  • The tally counts lines in the two files. A row "exact file rules" was planned and left out: ping has two (/etc/modules.conf, the /proc path) and curl has four (.curlrc, .config/curlrc, /etc/rpmdevtools/curlrc, @{run}/snapd.socket), so the number does not separate the styles.
  • The precise group in numbers (rule lines, abstractions, capability lines, lines with **, owner lines):
FileLinesAbstractionsCapabilities**ownerFlags
bin.ping153200none
sbin.klogd181200none
unix-chkpwd222202attach_disconnected.path=/att/unix-chkpwd/
usr.sbin.ntpd584900attach_disconnected
usr.sbin.dnsmasq884715attach_disconnected
usr.sbin.dovecot667941attach_disconnected
usr.sbin.smbd65812100attach_disconnected.path=/att/smbd/
curl, for contrast236013none
  • A larger example of the same style: ntpd has nine capabilities and 38 file rules, each a single file or a one-level pattern, and no ** (usr.sbin.ntpd:22-69).
  • The extras passwd profile has one rule per file the program rewrites: /etc/shadow rwl, /etc/shadow.old rwl, /etc/shadow.tmp?????? rwl, /etc/shadow.lock rwl, /etc/shadow- rw, /etc/shadow+ rw, (extras/usr.bin.passwd:23-37).

03 Capabilities, one by one, each with a reason

iotop-c SETS AN I/O PRIORITY iotop-c ioprio_set(...) the kernel asks, in this order 1 sys_admin ? 2 sys_nice ? # ... sys_nice covers the perms # required for this syscall and is less broad than sys_admin, # so silence a denial of sys_admin and force reliance on sys_nice. deny capability sys_admin, capability sys_nice, refused, not logged allowed: the priority is set grant + reason dnsmasq: capability net_admin, # for DHCP server decline fusermount3: # capability dac_override, isolate nvidia_modprobe: capability sys_module, only in its child kmod apparmor="DENIED" operation="capable" ... profile="iotop-c" pid=7292 comm="iotop-c" capability=21 capname="sys_admin" what the log would show without the deny line (values are illustrative)
iotop-c calls ioprio_set. The kernel asks for sys_admin first and sys_nice second. The profile has deny capability sys_admin and capability sys_nice with a comment giving the reason, so the first question is refused without a log record and the second is allowed. Three techniques with shipped examples: grant with a reason, decline, isolate in a child profile. At the bottom the DENIED record that would appear without the deny line.

deny capability sys_admin, plus capability sys_nice, (iotop-c) forces the weaker capability and silences the probe. In the kernel a capability without a rule fails with EPERM; with deny it fails without a log record. Read from the kernel code.

Iotop wants to set a real-time I/O priority. In the kernel, that call checks for sys_admin first and falls back to sys_nice, and sys_admin is nearly root. The profile's author read that code and wrote it down: deny sys_admin, allow sys_nice. The deny does two jobs: the kernel's first question gets a no without a log line, and no include can grant it back. So grant the weakest capability that works and say why, like dnsmasq's net_admin for the DHCP server. Decline what the program survives without, as fusermount does. And give a dangerous one only to the helper that needs it.

Details and sources

Details

  • Syntax: capability <name> [<name>...], and deny capability <name>,.
  • iotop-c: the comment and the two rules are at iotop-c:10-20. The slide quotes the end of the comment; the start says that ioprio_set checks sys_admin or sys_nice, "with sys_admin checked first, with OR shortcut logic", when it sets a real-time class.
  • That order is in the kernel: if (!capable(CAP_SYS_ADMIN) && !capable(CAP_SYS_NICE)) return -EPERM; for the real-time class (ioprio.c:47).
  • deny capability X, and no rule for X both refuse X. The deny form adds two things: no log record, and no later rule or include can grant it. In the kernel a capability without a rule fails with EPERM (capability.c:150-154); one in the quiet set, which is where deny puts it, returns the error without a record (capability.c:87-92).
  • The record at the bottom is built, not captured. Its layout is that of testcase_dmesg_capability.in (operation="capable" ... capability=23 capname="sys_nice"); profile, pid and comm are illustrative, and ... stands for the fixture's parent= field. sys_admin is capability 21 and sys_nice is 23 (capability.h:281, capability.h:296).
  • Grant with a reason: capability net_admin, # for DHCP server and capability net_raw, # for DHCP server ping checks (usr.sbin.dnsmasq:22-28).
  • Decline: fusermount3 keeps # capability dac_override, and # capability setuid, commented out under "Flatpak seems to work fine without these capabilities, so don't grant them unless actual functionality breakage is caused elsewhere" (fusermount3:11-15).
  • Isolate: nvidia_modprobe holds chown, mknod, setuid and sys_admin; capability sys_module, exists only in its kmod child (nvidia_modprobe:12-15, nvidia_modprobe:44).
  • More silencing: deny capability net_admin, # noisy setsockopt() calls (usr.sbin.traceroute:20), deny capability block_suspend, (usr.sbin.nscd:22), deny capability sys_admin, with "libudev seems to work fine without the BPF filter in place, so we can silence the denial" (lsusb:43-45).
  • The bare form capability, grants every capability (plasmashell:8).
  • aa-logprof shows a severity from severity.db with each capability question (CAP_SYS_ADMIN 10, CAP_SYS_MODULE 10, CAP_DAC_OVERRIDE 9, CAP_SETUID 9, CAP_NET_RAW 8). The number informs the author; the tool refuses nothing by severity.
  • What the log hides about capabilities in the kernel: the same capability for the same credentials is not logged again within one second (capability.c:70, capability.c:94-106), and optional probes are logged only in complain mode, with info="optional: no audit" (capability.c:139-144, capability.c:156-163).
  • Nothing on this slide was observed on a running kernel; kernel behaviour is read from the kernel source, profiles from AppArmor 836052d21 (6.0.0~alpha2).

for AppArmor developers for security researchers

Code

  • profile_capable: capability.c:121-166; the quiet test in audit_caps: capability.c:87-92; the hook apparmor_capable: lsm.c:199-211.
  • Which branch of profile_capable a profile compiled with abi <abi/5.0> takes (policy DFA or the older bit sets) was not determined; both refuse a capability that has no rule.
  • The severity of the bare capability, rule is that of __ALL__: capability.py:127-129.

04 Exact paths, and the one place for two stars

dnsmasq: FILE RULES dnsmasq 7 capabilities /etc/dnsmasq.d/ r, /etc/dnsmasq.d/* r, the directory, then its direct entries /var/lib/misc/dnsmasq.leases rw, # Required only for DHCP server usage the lease file, and why it is there @{run}/*dnsmasq*.pid w, a name that varies: one star, one directory profile dnsmasq /usr/{bin,sbin}/dnsmasq two layouts: an alternation, not a glob @{TFTP_DIR}/ ** r , the only two stars: read-only, for the TFTP server the price: a block of rules for each integration: libvirt, lxc-net, lxd, NetworkManager, podman, waydroid
dnsmasq, which holds seven capabilities, and five of its file rules: a directory rule and a rule for its direct entries, one lease file with a comment, a pid file pattern with single stars, the profile line with an alternation for bin and sbin, and the only rule with two stars, which is read-only for the TFTP directory. A foot line lists the integrations that each have a block of rules.

dnsmasq: /etc/dnsmasq.d/ r, /etc/dnsmasq.d/* r, /var/lib/misc/dnsmasq.leases rw, @{run}/*dnsmasq*.pid w, and one read-only **: @{TFTP_DIR}/** r,. Site additions go in local/usr.sbin.dnsmasq.

Dnsmasq holds seven capabilities, so its file rules are careful. Configuration: the directory and its direct entries, two rules, read only. The lease file, read and write, with a note that only the DHCP server needs it. Pid files: a star for the part of the name that varies, inside one directory. The binary lives in bin on some systems and sbin on others, so the path is an alternation, not a wildcard. And the only double star is read only, for the TFTP directory. The price is upkeep: every program that integrates with dnsmasq has its own block of rules, and a new one needs a new block.

Details and sources

Details

  • * matches within one path component, ** crosses /. A rule for a directory itself ends in /: /etc/dnsmasq.d/ r, lists the directory, /etc/dnsmasq.d/* r, reads its direct entries (usr.sbin.dnsmasq:42-43).
  • The other rules on the slide: /var/lib/misc/dnsmasq.leases rw, # Required only for DHCP server usage (usr.sbin.dnsmasq:66), @{run}/*dnsmasq*.pid w, (usr.sbin.dnsmasq:61), the profile line with /usr/{bin,sbin}/dnsmasq (usr.sbin.dnsmasq:17), and @{TFTP_DIR}/** r, under "for the read-only TFTP server" (usr.sbin.dnsmasq:74-76).
  • @{TFTP_DIR}=/var/tftp /srv/tftp /srv/tftpboot is declared in the profile file itself (usr.sbin.dnsmasq:14).
  • "Seven capabilities" counts usr.sbin.dnsmasq:22-28.
  • The integration blocks: libvirt, lxc-net, lxd-bridge, NetworkManager, podman's dnsname plugin and waydroid (usr.sbin.dnsmasq:78-119). Several of them add further lease and pid files, such as /var/lib/misc/dnsmasq.*.leases rw,.
  • Site additions go through include if exists <local/usr.sbin.dnsmasq> (usr.sbin.dnsmasq:142), so the reviewed file stays unchanged.
  • More exact paths in shipped profiles: /dev/nvidia-modeset w, /dev/nvidia-uvm w, /dev/nvidia-uvm-tools w, (nvidia_modprobe:27-29); @{run}/mount/utab.lock rwk, (fusermount3:55); /var/log/wtmp.@{d} r, for rotated logs (who:11); /usr/lib{,64}/libvirt/libvirt_leaseshelper (usr.sbin.dnsmasq:87).
  • In aa-logprof the (N)ew key lets you type the exact pattern; the (G)lob key works against this style (aa.py:1248-1287).
  • Chroots: ntpd lists the same files below its chroot, /var/lib/ntp/@{PROC}/@{pid}/net/if_inet6 r, and /var/lib/ntp@{run}/ntp/ntpd.pid w, (usr.sbin.ntpd:57, usr.sbin.ntpd:72-73), because the name the kernel reports depends on where the task's root is.

05 Every program it starts gets a profile

EXEC: WHERE DOES THE PRIVILEGE GO? nvidia_modprobe capabilities: chown, mknod, setuid, sys_admin /usr/bin/kmod Cx -> kmod, exec kmod nvidia_modprobe//kmod capability sys_module, /usr/bin/kmod mrix, /etc/modprobe.d/{,*.conf} r, loading kernel modules is possible only inside the child dnsmasq /usr/libexec/libvirt_leaseshelper Cx -> libvirt_leaseshelper, dnsmasq//libvirt_leaseshelper no capabilities, 9 file rules /{,usr/}bin/{ba,da,}sh ix , for --dhcp-script the script runs with dnsmasq's profile and its 7 capabilities dovecot: 19 helpers, each Px to its own profile file
The nvidia_modprobe profile holds four capabilities and runs /usr/bin/kmod with Cx into its child profile kmod, the only place with capability sys_module. The dnsmasq profile runs libvirt_leaseshelper with Cx into a child profile with no capabilities and nine file rules, and runs the shell with ix, so a script keeps dnsmasq's profile and its seven capabilities.

Cx -> child moves a helper into a child profile: capability sys_module, exists only in nvidia_modprobe's kmod child. ix keeps the parent's profile, capabilities included: /{,usr/}bin/{ba,da,}sh ix, in dnsmasq.

A privileged program that starts another program hands something on. The question is what. nvidia_modprobe needs a kernel module loaded, and that takes sys_module, about the most dangerous capability there is. The profile doesn't hold it. It runs kmod in a child profile, and only the child has sys_module. Dnsmasq does the same for a libvirt helper: a child with no capabilities and nine file rules. Now the next line. The shell runs with ix, inherit, so a DHCP script runs with dnsmasq's whole profile, seven capabilities included. The comment says why. Somebody made that decision, and in a precise profile every exec line is one.

Details and sources

Details

  • Forms: <path> Px, (the target's own profile), <path> Px -> name, (a named profile), <path> Cx -> child, (a child profile in the same file), <path> ix, (stay in the current profile). An upper-case P or C asks for secure execution, which scrubs the environment; see safe and unsafe exec.
  • nvidia_modprobe: four separate capability lines, summarised on the slide (nvidia_modprobe:12-15); /usr/bin/kmod Cx -> kmod, (nvidia_modprobe:23); the child with capability sys_module, (nvidia_modprobe:39-62). A child profile is named parent//child in the kernel, hence nvidia_modprobe//kmod.
  • dnsmasq: /usr/libexec/libvirt_leaseshelper Cx -> libvirt_leaseshelper, (usr.sbin.dnsmasq:87-88); the child has one abstraction, no capability and nine file rules (usr.sbin.dnsmasq:121-139).
  • /{,usr/}bin/{ba,da,}sh ix, # Required to execute --dhcp-script argument (usr.sbin.dnsmasq:68). With ix the shell and the script keep the profile dnsmasq, including usr.sbin.dnsmasq:22-28. This follows from the rule; it was not run.
  • dovecot: 19 helpers with Px, each with its own profile file, and two with rix (usr.sbin.dovecot:50-70).
  • Several children with different rights: zgrep sends the compressors and grep to helper, sed to sed, and runs cat, expr, mktemp, rm and the shell with ix (zgrep:23-37, zgrep:46-73).
  • A named Px across files: /usr/lib*/samba/{,samba/}samba-dcerpcd Px -> samba-dcerpcd, (usr.sbin.smbd:43).
  • Programs that switch profile themselves: php-fpm has change_profile -> php-fpm//*, and write access to @{PROC}/@{pid}/attr/{apparmor/,}current (php-fpm:29, php-fpm:50-57); apache2 uses hats (usr.sbin.apache2:90-103).
  • aa-logprof offers inherit, profile, child, a named target, unconfined and deny for each exec event (aa.py:622-661); the child choice is not offered for an exec seen inside a child profile or hat (aa.py:777-779).
  • The only ux-family rule in profiles/apparmor.d/ is /** pux, in plasmashell (plasmashell:24), one of the allow-everything profiles.

06 Subtract, name the peer, and log your guesses

QUALIFIERS AT WORK include <abstractions/wutmp> audit deny /var/** w, audit deny @{run}/utmp w, who /var/log/wtmp r w who: "Deny the writes allowed by abstractions/wutmp" signal (receive) peer=libvirtd, ptrace (readby) peer=libvirtd, libvirtd dnsmasq signal anything else # fi.w1.wpasupplicant.Group signals (unknown path) audit dbus (send) bus=system path=/fi/w1/wpa_supplicant1/** ... allowed, and logged the author was not sure of the path: allow it, and get a record each time it is used plain deny is silent. audit deny tells you somebody tried.
Three bands. The who profile includes abstractions/wutmp and then has two audit deny rules for writes, so who can read /var/log/wtmp and cannot write it. The dnsmasq profile has signal receive and ptrace readby rules with peer=libvirtd, so libvirtd can signal dnsmasq and anything else cannot. The wpa_supplicant profile has an audit dbus send rule under a comment that says the path is unknown: it is allowed and logged each time it is used.

audit deny /var/** w, after include <abstractions/wutmp> (who) removes a grant. signal (receive) peer=libvirtd, (dnsmasq) names the sender. audit on an allow rule (wpa_supplicant) logs each use.

Three small tools that precise profiles lean on. First, subtraction. Who includes the login records abstraction for its list of files, but that abstraction also allows writing them. So the profile denies the writes again, with audit, so an attempt shows up in the log. Second, peers. Dnsmasq accepts signals, and being inspected, but only from libvirtd, by name. Third, an honest guess. The author of the wpa_supplicant profile wasn't sure about one D-Bus path, so the rule allows it with audit in front, and every use leaves a record. One thing to remember: a plain deny is silent. If you want to hear about attempts, write audit deny.

Details and sources

Details

  • Qualifier order: [audit] [deny] [owner] <rule>.
  • who: include <abstractions/wutmp> (who:9), then "Deny the writes allowed by abstractions/wutmp" with audit deny /var/** w, and audit deny @{run}/utmp w, (who:39-41).
  • dnsmasq: signal (receive) peer=libvirtd, and ptrace (readby) peer=libvirtd,, each also written with the path-named peer /usr/{bin,sbin}/libvirtd (usr.sbin.dnsmasq:32-35). The peer is a profile name. The rule covers any signal; the slide shows one.
  • wpa_supplicant: the audit dbus (send) rule spans five lines in the file and is shortened with ... on the slide (wpa_supplicant:84-89). It is a D-Bus rule: the bus daemon enforces and logs it, not the kernel. See D-Bus mediation.
  • audit appears in four files of profiles/apparmor.d/ (tnftp, unprivileged_userns, wpa_supplicant, who); three of them use it only as audit deny.
  • A plain deny writes no record; audit deny does. See The audit qualifier.
  • peer=<profile> and peer=(label=<profile>) are accepted on signal, ptrace, unix and dbus rules. More shipped examples: unix (receive, send) type=stream peer=(label=dovecot-anvil), (usr.sbin.dovecot:39-40); signal (send) peer=@{profile_name}//*, (usr.sbin.apache2:76); a unix rule tied to an address, addr="@/tmp/.X11-unix/X[0-9]*" (Xorg:65).
  • Deny for what must not happen at all: audit deny capability, and audit deny change_profile, (unprivileged_userns:8-9); deny network, in tnftp's command child (tnftp:77).
  • Deny to silence a harmless probe: deny /usr/bin/apt-cache x, and deny /tmp/gtalkplugin.log w, (lsb_release:43, lsb_release:52); deny / rw, (php-fpm:47).
  • owner inside privileged profiles: unix-chkpwd:37-39; for rootless podman, next to the unqualified root-owned form, usr.sbin.dnsmasq:111-116.
  • A deny cannot be undone by a later allow of the same priority, local/ included.
  • aa-logprof cannot add a deny through an include: "Denying via an include file isn't supported by the AppArmor tools" (aa.py:1232-1234).
  • Known discrepancy: an audit deny file rule in a complain-mode profile appears to be allowed, going by the kernel code (verdict unclear, not run). who relies on audit deny to cancel the writes from wutmp. Test deny rules in enforce mode.
  • Known discrepancy: zgrep has both /etc/passwd r, and deny /etc/passwd r, (zgrep:22, zgrep:42). The deny decides and the allow is dead. Read from the file, not run.

07 Finding every operation: the log is not enough

WHAT DOES THE PROGRAM REALLY DO? $ sudo aa-complain who apparmor="ALLOWED" operation="open" ... profile="who" name="/dev/pts/" ... requested_mask="r" denied_mask="r" ... one record per missing permission; the run continues $ sudo aa-audit who log every operation the profile allows, too # ... dirfd=openat(AT_FDCWD, "/dev", # O_RDONLY|O_PATH|O_DIRECTORY) # and newfstatat(dirfd, terminal_name, # out_ptr) # - O_PATH fds bypass mediation so /dev/ r, # is not needed for the first step # Overall result: the above calls are not # mediated by LSMs a comment in the shipped who profile: system calls and arguments, not log lines the log will not show code paths you did not run denials silenced by a deny rule a capability check repeated within 1 s why the program does it an empty log proves those runs needed nothing more, not that the profile is complete
Left: what the log gives. aa-complain who and an ALLOWED record for /dev/pts/, then aa-audit who to log allowed operations too. Right: a comment from the shipped who profile that quotes an openat call with O_PATH and newfstatat and concludes that the calls are not mediated by LSMs. Below, four things the log will not show: code paths you did not run, denials silenced by a deny rule, a capability check repeated within one second, and why the program does it.

Complain-mode records and aa-audit show what reached AppArmor during your runs. Shipped comments (who, iotop-c, lsusb) cite system calls, kernel code and library source: knowledge from tracing and reading code, which the log does not contain. That an O_PATH open is not mediated is read from the kernel code.

How did the author know all that? The AppArmor side is the log. In complain mode each missing permission becomes a record while the program keeps running, and aa-audit logs what the profile allows. But read the comment in the who profile. It quotes an openat call with O_PATH, and concludes those calls aren't mediated. No log line says that. It comes from system calls and man pages. The log also can't show code you didn't run, denials a deny rule silenced, or why the program wants a file. So you need both: the log, and the program itself, through a tracer such as strace and its source.

Details and sources

Details

  • The record on the slide is built, not captured: the field layout is that of syslog_datetime_01.in (apparmor="ALLOWED" operation="open" parent=... profile=... name=... pid=... comm=... requested_mask="r" denied_mask="r" fsuid=... ouid=...), with illustrative values. It is the record that /dev/pts/ r, in the who profile answers (who:27).
  • File records carry operation=, profile=, name=, requested_mask=, denied_mask=, fsuid=, ouid=; capability records carry capability= and capname=. Reading them is in Reading a denial.
  • The comment on the right is who:14-24, wrapped for the panel. It ends "Overall result: the above calls are not mediated by LSMs", and the profile has no /dev/ r, rule.
  • In the kernel an O_PATH open returns before the file-open security hook is called (open.c:951-956, open.c:977). Read from the code, not run. See What is not mediated.
  • aa-audit adds the audit flag to the profile and reloads it ("Setting %s to audit mode.", tools.py:228-243). aa-audit --remove takes it away again ("Removing audit mode from %s."). In the kernel every allowed permission of such a profile is logged as an AUDIT record, and denials that deny rules keep quiet are logged too. aa-logprof drops AUDIT records, so they are read by a person. No shipped profile carries the flag.
  • The rule classes aa-logprof turns into rules: capability, change_profile, dbus, file, ptrace, signal, userns, mqueue, io_uring, mount, unix, network, pivot_root (logparser.py:61-75).
  • aa-unconfined "Lists unconfined processes having tcp or udp ports" (aa-unconfined:31): a way to find network-facing programs that have no profile yet.
  • Not in the log, the complete list: operations already allowed (unless the audit flag or an audit rule is set); denials quieted by deny rules; a capability check repeated for the same credentials within one second; optional capability probes in enforce mode; records lost to rate limiting (aa-genprof sets /proc/sys/kernel/printk_ratelimit to 0 and warns "Can't set printk_ratelimit, some events might be lost" when it cannot, aa-genprof:133-143; aa-logprof alone does not); code paths the runs did not reach; operations that are not mediated; and the reason for any of it.
  • Where shipped comments got their reasons: system-call arguments (who:19-20), kernel source (iotop-c:11-18), library source files by name (lsusb:40-41), program options (curl:30-32, usr.sbin.dnsmasq:68), bug reports (fusermount3:11).
  • Neither tree contains or invokes a tracer. Tracing and source reading are recommended here as practice that these comments give evidence of. Which tools the authors used was not determined.

for AppArmor developers for security researchers

Code

08 Privileged and broad: what to look for in a shipped profile for security researchers

WHERE PRECISION ENDS tar: capability dac_override, file rwl /**, DAC bypass on every path, links included syslogd: capability dac_override, /var/log/** rw, any file under /var/log, any owner apache2 (and both hats): /** mrwlkix, every file allowed: DAC decides dnsmasq: /{,usr/}bin/{ba,da,}sh ix, + 7 capabilities a script inherits all 7 capabilities extras sshd: /{usr/,}bin/bash Uxr, ... 14 such rules by design: the session leaves the profile $ apparmor_parser -p PROFILE | grep -E 'capability|/\*\* |[UuPp]*x,| ix,' start an audit here: expand the includes, then look for these combinations
Five shipped rule combinations and what each allows. tar: capability dac_override with file rwl on every path. syslogd: dac_override with read and write on all of /var/log. apache2 and both hats: every file with mrwlkix. dnsmasq: a shell with ix next to seven capabilities. The extras sshd profile: Uxr for login shells, by design. At the bottom a command that expands a profile and greps it for capabilities, two-star rules and exec modes.

Shipped examples of privilege next to breadth: tar (dac_override, file rwl /**,), sbin.syslogd (dac_override, /var/log/** rw,), usr.sbin.apache2 (/** mrwlkix,), usr.sbin.dnsmasq (sh ix with seven capabilities), extras usr.sbin.sshd (Uxr for shells). What each allows is derived from the rules, not demonstrated.

Precise is the goal, and the tree doesn't always reach it. Look for these combinations. A capability that bypasses file permissions, next to a whole-tree rule: tar has dac_override with read, write and link on everything, and syslogd has it with all of /var/log. A profile whose main rule is /**, like apache's. A shell run with inherit by a daemon that holds capabilities, as in dnsmasq. And unconfined exec, which the extras sshd profile uses for login shells on purpose. None of these is a kernel bug. Each is that profile's real boundary. So expand the profile and grep before you trust its name.

Details and sources

Attack surface

Each consequence below is derived from the rule text. None was demonstrated on a running system.

  • tar: capability dac_override, and capability dac_read_search, under the comment "used to compress user files as root", then file rwl /**, (tar:18-25). The file rule covers every path, link permission included, and the capability removes the permission-bit check. The same profile runs any program in the bin directories with ix (tar:27-30), which keeps those capabilities. What remains as a control: the profile's lack of other capabilities and of network rules.
  • syslogd: capability dac_override, (sbin.syslogd:22) next to /var/log/** rw, (sbin.syslogd:42): any file under /var/log, whoever owns it.
  • apache2: six capabilities, dac_override among them, with / rw, and /** mrwlkix, (usr.sbin.apache2:78-86); the hats DEFAULT_URI and HANDLING_UNTRUSTED_INPUT have the same file rule (usr.sbin.apache2:95, usr.sbin.apache2:102). The profile gives the process a name and a capability list; file access is left to the permission bits.
  • dnsmasq: /{,usr/}bin/{ba,da,}sh ix, # Required to execute --dhcp-script argument (usr.sbin.dnsmasq:68) next to seven capabilities (usr.sbin.dnsmasq:22-28), net_admin, dac_override and setuid among them. A script run this way has the whole profile.
  • extras sshd: 14 Uxr rules for shells and for nologin and false (extras/usr.sbin.sshd:92-105), /usr/lib/openssh/sftp-server PUx, (extras/usr.sbin.sshd:160), fifteen capabilities (extras/usr.sbin.sshd:29-43) and /** r, with owner /** rwl, (extras/usr.sbin.sshd:153-155). For a login service the session is meant to leave the daemon's profile. The extras directory holds profiles that are not enabled by default.
  • Side grants inside precise profiles: ping, dnsmasq, ntpd, unix-chkpwd and nscd include abstractions/nameservice, which adds inet stream and dgram sockets, network netlink raw, and capability net_bind_service, (nameservice:92-99, nis:16).
  • The l permission in world-writable directories, in a root daemon: /tmp/ntp* rwl, and /var/tmp/ntp* rwl, (usr.sbin.ntpd:43, usr.sbin.ntpd:68). See Hard links and name-based rules.
  • A loaded profile name is not evidence of confinement: of the 198 profile files directly in profiles/apparmor.d/, 76 are flags=(unconfined) and 14 are allow all,.
  • aa-logprof -a accepts every logged event and is a source of weak generated profiles.
  • Known discrepancy: with aa-logprof -a the secure-execution question is answered "no", so an exec to a profiled target is written as lower-case px, although the interactive default is Px (ui.py:159-160, ui.py:518-526, aa.py:884-887). Read from the code, not run.

Details

  • apparmor_parser -p FILE (--preprocess) prints the profile with its includes expanded, which is what the grep on the slide needs. The pattern is an illustration of where to look; no output of it was recorded for the map.
  • aa-status lists the loaded profiles and their modes; see aa-status and aa-enabled.

09 From complain to enforce, the careful way

SIX STEPS 1 generate $ sudo aa-genprof --no-abstraction /usr/sbin/dnsmasq paths, not include offers 2 exercise, then (S)can DHCP, TFTP, reload, shutdown, error paths 3 answer (N)ew: type the exact path exec: px or cx, secure: yes not (G)lob, not ix by default 4 edit by hand # reasons deny ..., audit deny ..., peer=... the tools do not write these 5 enforce, then watch $ sudo aa-enforce dnsmasq $ sudo aa-audit dnsmasq what is used, what is refused 6 site additions local/usr.sbin.dnsmasq the reviewed file stays as it is aa-logprof -a accepts every event: the opposite of this procedure
Six steps in two rows. One: aa-genprof with --no-abstraction on /usr/sbin/dnsmasq. Two: exercise everything, then Scan. Three: answer with New and the exact path, and px or cx with secure execution. Four: edit by hand to add reasons, deny rules and peers. Five: aa-enforce, then aa-audit. Six: site additions in local/usr.sbin.dnsmasq. A warning strip: aa-logprof -a accepts every event, the opposite of this procedure.

aa-genprof --no-abstraction, (N)ew for exact paths, px/cx with secure execution, hand-written comments, deny and peer= rules, aa-enforce, then aa-audit for a watch period. Site changes go into local/<profile>.

The tools are the ones from the general-purpose deck. Your answers change. Start aa-genprof with the --no-abstraction option, so it offers paths instead of includes. Exercise everything, including reload, shutdown and error paths. For each question, press N and type the exact path instead of globbing. For each exec, choose a profile or a child, and say yes to secure execution. Then edit by hand: reasons, denies, peers. The tools don't write those. Enforce, and run with aa-audit to see what the profile uses and refuses. Keep site changes in the local file. And stay away from aa-logprof -a, which accepts everything.

Details and sources

Details

  • aa-genprof [-d DIR] [-f LOGFILE] [-j] [--no-abstraction] [-o DIR] program. --no-abstraction: "Do not use any abstractions in profiles" (aa-genprof:68-73). The new profile starts in complain mode; Finish switches it to enforce (aa-genprof:187-190).
  • aa-logprof [-d DIR] [-f LOGFILE] [-m MARK] [-j] [-a] [--no-abstraction] [-o DIR]. -a, --allow-all: "Accept silently all rules" (aa-logprof:27-33).
  • aa-enforce, aa-complain and aa-audit take program names, or --profile, --profile-file, --executable-file, plus -d DIR and --no-reload. aa-enforce and aa-complain skip a flags=(unconfined) profile without --force ("Profile %s is in unconfined mode. Use --force to switch it to enforce mode."). aa-audit -r (--remove) removes the audit flag again (tools.py:196-243). aa-enforce clears the complain flag and leaves an audit flag in place (aa.py:276-281).
  • (N)ew asks for the path and warns when it does not match the log entry: "The specified path does not match this log entry" (aa.py:1260-1287).
  • After px or cx the tool asks "Should AppArmor enable secure-execution mode when switching profiles?" with yes as the default; yes gives Px or Cx. After ux it warns "Launching processes in an unconfined state is a very dangerous operation and can cause serious security holes." (aa.py:863-900).
  • The tools do not write comments, peer conditions or audit deny rules. Those are edits by hand.
  • Long-running daemons in the tree carry attach_disconnected (ntpd, dnsmasq, dovecot) or attach_disconnected.path= (smbd, unix-chkpwd), and the second form needs rules for the prefixed paths (unix-chkpwd:41-46). See The other profile flags.
  • Not determined: whether aa-genprof's Finish clears complain on cx child profiles created during the session. Check the flags of child profiles in the written file.
  • Test deny rules in enforce mode. Known discrepancy: an audit deny file rule in a complain-mode profile appears to be allowed, going by the kernel code (verdict unclear, not run).
  • Known discrepancy: with aa-logprof -a an exec to a profiled target is written as lower-case px although the interactive default is Px (ui.py:159-160, ui.py:518-526). Read from the code, not run.

10 Three reviews before you enforce

ROADS your precise profile symlinks /{usr/,}bin/{,iputils-}ping does the rule name the path the kernel resolves? prac-symlinks hard links /tmp/ntp* rwl, what can be linked there, and by whom? prac-hardlinks /proc and /sys @{sys}/devices/@{pci_bus}/ **/config r, self, pids and device paths prac-sys-proc syn-capability syn-child-profiles syn-audit prac-abstractions sec-unmediated adm-denials the strict variants what no rule can cover reading the records prac-general-purpose: when all this care is not needed
Your precise profile in the centre with three gates below it: symlinks with the ping profile's path and a road to prac-symlinks, hard links with ntpd's /tmp/ntp* rwl rule and a road to prac-hardlinks, and /proc and /sys with a device path from nvidia_modprobe and a road to prac-sys-proc. Below, signs to prac-abstractions, syn-capability, syn-child-profiles, syn-audit, sec-unmediated and adm-denials, and a foot line naming prac-general-purpose.

Review a precise profile for symlinks (prac-symlinks), hard links (prac-hardlinks) and /proc and /sys paths (prac-sys-proc). Then: prac-abstractions, syn-capability, syn-child-profiles, syn-audit, sec-unmediated, adm-denials.

Before you enforce a precise profile, review three things, each with its own deck. Symlinks: an exact rule has to name the path the kernel ends up at, which is why ping's profile spells out both bin directories. Hard links: the link letter in a directory other users can write, like ntpd's temp file rule, deserves a second look. And /proc and /sys, where the path your program opens is often not the path that gets checked. After those, the decks on capabilities, child profiles and audit go deeper. And if your program holds no privileges at all, the general-purpose deck is the easier road.

Details and sources

Details

Read more

Basis of the statements on this page: profile text is quoted from the AppArmor tree at 836052d21 (6.0.0~alpha2); "code" means the source of mainline Linux 7.4 (97d73c825) or of the AppArmor utilities was read, not run; "derived" means the result was reasoned from that code or from rule text. Nothing here was observed on a running kernel.

Precise profiles in the tree

Counts per file in profiles/apparmor.d/: rule lines, include <abstractions/...> lines, capability lines, lines containing **, owner lines.

FileLinesAbstractionsCapabilities**ownerFlags
bin.ping153200none
sbin.klogd181200none
unix-chkpwd222202attach_disconnected.path=/att/unix-chkpwd/
usr.sbin.ntpd584900attach_disconnected
usr.sbin.dnsmasq884715attach_disconnected
usr.sbin.dovecot667941attach_disconnected
usr.sbin.smbd65812100attach_disconnected.path=/att/smbd/

Privileged and broad: capabilities next to a whole-tree rule.

ProfileRulesWhere
tarcapability dac_override, and file rwl /**,tar:18-25
sbin.syslogdcapability dac_override, and /var/log/** rw,sbin.syslogd:22, sbin.syslogd:42
usr.sbin.apache2 and its two hatssix capabilities and /** mrwlkix,usr.sbin.apache2:78-86
zgrep, child helpercapability dac_override, capability dac_read_search, and /{,**} r,zgrep:50-51, zgrep:62
extras usr.sbin.sshdfifteen capabilities, /** r,, owner /** rwl,, Uxr for shellsextras/usr.sbin.sshd:29-43, extras/usr.sbin.sshd:92-105, extras/usr.sbin.sshd:153-155

Of the 198 profile files directly in profiles/apparmor.d/, 76 are flags=(unconfined) and 14 are allow all,.

Capability techniques

TechniqueShipped exampleWhere
Grant with a reasoncapability net_admin, # for DHCP serverusr.sbin.dnsmasq:22-28
Decline# capability dac_override, kept commented outfusermount3:11-15
Choose the weakerdeny capability sys_admin, with capability sys_nice,iotop-c:10-20
Silencedeny capability net_admin, # noisy setsockopt() callsusr.sbin.traceroute:20
Isolate in a childcapability sys_module, only in nvidia_modprobe//kmodnvidia_modprobe:44

Kernel results (code):

  • A capability with no rule fails with EPERM.
  • A capability under deny fails with EPERM and no record, unless the profile has the audit flag.
  • The same capability for the same credentials is not logged again within one second.
  • Optional probes are not logged in enforce mode; in complain mode they are logged with info="optional: no audit".
  • The bare rule capability, grants every capability.

Path techniques

TechniqueExample from dnsmasqLine
Directory, then its entries/etc/dnsmasq.d/ r, and /etc/dnsmasq.d/* r,usr.sbin.dnsmasq:42-43
A single state file/var/lib/misc/dnsmasq.leases rw,usr.sbin.dnsmasq:66
One-star pattern in one directory@{run}/*dnsmasq*.pid w,usr.sbin.dnsmasq:61
Alternation for two layouts/usr/{bin,sbin}/dnsmasqusr.sbin.dnsmasq:17
Read-only **@{TFTP_DIR}/** r,usr.sbin.dnsmasq:74-76
Variable declared in the profile@{TFTP_DIR}=/var/tftp /srv/tftp /srv/tftpbootusr.sbin.dnsmasq:14
Site additionsinclude if exists <local/usr.sbin.dnsmasq>usr.sbin.dnsmasq:142

Exec decisions

ModeMeaningShipped exampleWhat the started program has
Pxthe target's own profile, secure executiondovecot's 19 helpers (usr.sbin.dovecot:50-70)only its own profile
Px -> namea named profile, secure executionsamba-dcerpcd Px -> samba-dcerpcd, (usr.sbin.smbd:43)the named profile
Cx -> childa child profile in the same file/usr/bin/kmod Cx -> kmod, (nvidia_modprobe:23)the child's rules only
ixstay in the current profile/{,usr/}bin/{ba,da,}sh ix, (usr.sbin.dnsmasq:68)everything the parent has, capabilities included
Ux, PUxleave confinement, secure executionshells in the extras sshd profile (extras/usr.sbin.sshd:92-105)no profile

Qualifiers in privileged profiles

UseExampleWhere
audit deny to subtract from an includeaudit deny /var/** w, after include <abstractions/wutmp>who:39-41
deny for what must not happenaudit deny capability, and audit deny change_profile,unprivileged_userns:8-9
deny to silence a probedeny /usr/bin/apt-cache x,lsb_release:43
audit on an allow ruleaudit dbus (send) bus=system path=/fi/w1/wpa_supplicant1/** ...wpa_supplicant:84-89
ownerowner /dev/pts/[0-9]* rw,unix-chkpwd:37-39
Peer by profile namesignal (receive) peer=libvirtd,usr.sbin.dnsmasq:32-35
Peer by labelunix (receive, send) type=stream peer=(label=dovecot-anvil),usr.sbin.dovecot:39-40
  • A plain deny is silent; audit deny logs the attempt.
  • A deny cannot be undone by a later allow of the same priority, local/ included.

What the log shows and what it does not

Shows:

  • one record per refused permission, or in complain mode per permission that would have been refused;
  • with audit on a rule, or after aa-audit on the profile, each use of an allowed permission as an AUDIT record (code).

Does not show:

  • operations that were already allowed;
  • denials quieted by deny rules;
  • a capability check repeated for the same credentials within one second (code);
  • optional capability probes in enforce mode (code);
  • records lost to rate limiting; aa-genprof sets printk_ratelimit to 0, aa-logprof alone does not;
  • code paths the runs did not reach: error handling, options, reload, shutdown;
  • operations that are not mediated, such as an O_PATH open (code);
  • why the program does any of it.

The procedure

StepCommand or keyTool message or note
1aa-genprof --no-abstraction /usr/sbin/dnsmasq"Do not use any abstractions in profiles"; the profile starts in complain mode
2exercise the program, then (S)caninclude reload, shutdown and error paths
3(N)ew for each path; profile or child for each exec"Should AppArmor enable secure-execution mode when switching profiles?": yes
4edit by handreasons as comments, deny, audit deny, peer=
5aa-enforce dnsmasq, then aa-audit dnsmasq"Setting %s to enforce mode.", "Setting %s to audit mode."; aa-audit --remove ends the watch: "Removing audit mode from %s."
6local/usr.sbin.dnsmasqread through include if exists <local/usr.sbin.dnsmasq>

aa-logprof -a ("Accept silently all rules") is the opposite of this procedure.

Review checklist

CheckShipped lineDeck
Symlinks: does the rule name the resolved path?/{usr/,}bin/{,iputils-}ping (bin.ping:15)Symlinks: rules see through them
Hard links: l where others can write/tmp/ntp* rwl, (usr.sbin.ntpd:43)Hard links and name-based rules
/proc and /sys: self, pids, device paths@{sys}/devices/@{pci_bus}/**/config r, (nvidia_modprobe:30-31)Writing rules for /sys and /proc
Chroots: paths as seen from the task's root/var/lib/ntp@{run}/ntp/ntpd.pid w, (usr.sbin.ntpd:57)Symlinks: rules see through them
Write globs next to DAC-bypass capabilities/var/log/** rw, with capability dac_override, (sbin.syslogd:42)The limits of name-based mediation

Known discrepancies

  • aa-logprof -a and px: in allow-all mode the secure-execution question is answered "no", so an exec to a profiled target is written as lower-case px although the interactive default is Px (ui.py:159-160, ui.py:518-526, aa.py:884-887). Read from the code, not run.
  • zgrep allows and denies the same file: /etc/passwd r, and deny /etc/passwd r, (zgrep:22, zgrep:42). The deny decides; the allow is dead. Read from the file, not run.
  • An audit deny file rule in a complain-mode profile appears to be allowed by the kernel (verdict unclear). who relies on audit deny to cancel writes from abstractions/wutmp. Test deny rules in enforce mode. Read from the code, not run.

for AppArmor developers for security researchers

Code

Roads from here

FILE ACCESS

QUALIFIERS

RULE CLASSES

PROFILE STRUCTURE

IN PRACTICE · general-purpose apps

IN PRACTICE · security-sensitive programs

SECURITY · threat model and limits

RUNNING THE PARSER