Reading a denial

what a DENIED line says for application developers for administrators

Every AppArmor decision that is logged is one record. It goes to the audit log when the audit daemon runs and to the kernel log otherwise. The first field says DENIED or ALLOWED or AUDIT or KILLED or STATUS. The others name the profile and the operation and the object and the refused permission and together they are the rule that would allow it. A plain deny rule and the rate limit of the kernel log both leave no line.

On the map: running the parser

▶ Watch this deck — 9 scenes, ~7 min

Worth seeing first

  • Why AppArmor — what a profile is and a first look at a denial
  • File rules — the path and the letters that a file record maps back to

Part of journeys

01 Permission denied: was it AppArmor?

READING A DENIAL $ aa-exec -p demo -- cat /etc/demo/secret.conf cat: /etc/demo/secret.conf: Permission denied apparmor="DENIED" operation="open" class="file" profile="demo"name="/etc/demo/secret.conf" pid=1323 comm="cat"requested_mask="r" denied_mask="r" fsuid=0 ouid=0 one line from the kernel profile="demo" which profile refused operation="open" doing what name="/etc/demo/secret.conf" to what where the line lands its first word its fields when there is no line /etc/demo/secret.conf r, the one rule that would allow it or the decision that the profile was right
demo runs cat on /etc/demo/secret.conf under the profile demo and gets Permission denied. The kernel writes one record: apparmor DENIED, operation open, class file, profile demo, name /etc/demo/secret.conf, pid, comm cat, requested_mask r, denied_mask r, fsuid 0, ouid 0. Three fields answer which profile refused, doing what, and to what. The deck covers where the line lands, its first word, its fields and when there is no line. The rule that would allow it is /etc/demo/secret.conf r.

A refused access is one record: the profile that refused, the operation, the object and the permission. This deck reads it field by field, for administrators and application developers.

Demo tries to open a file, and gets permission denied. The file's mode looks fine, so who said no? If it was AppArmor, the kernel wrote one line about it, and that line answers three questions: which profile refused, what the program was doing, and to what. This deck teaches you to read that line. We'll find where it lands, look at its first word, go through its fields, and see when a refusal leaves no line at all. By the end you can take a record and write the one rule that would allow it. Or decide that the profile was right.

Details and sources

Details

  • The command and the record are real. Logged by a Debian sid guest (kernel 7.3.4, a distribution kernel without Ubuntu's AppArmor patches); the field names and their order were checked against the kernel source the map describes. The profile demo was compiled by the parser of the AppArmor tree the map describes and entered with aa-exec -p demo; it allows reading /etc/demo/demo.conf and has no rule for /etc/demo/secret.conf.
  • The full line in the kernel log: audit: type=1400 audit(1794831779.488:174): apparmor="DENIED" operation="open" class="file" profile="demo" name="/etc/demo/secret.conf" pid=1323 comm="cat" requested_mask="r" denied_mask="r" fsuid=0 ouid=0. On the slide the prefix audit: type=1400 audit(SECONDS:SERIAL): is left out and the record is broken over lines.
  • comm="cat" and not "demo": the record names the command that ran under the profile. The profile is in profile=.
  • A program gets EACCES ("Permission denied") from AppArmor exactly as from file modes; the record is the only thing that tells the two apart.
  • The first look at a denial in the map is the scene "What a denial looks like" of Why AppArmor; its line is composed for the example, the ones in this deck were logged.

for AppArmor developers for security researchers

In the code

02 Where the line ends up

WHERE RECORDS GO AppArmor the kernel's audit system the audit daemon is running /var/log/audit/audit.log type=AVC msg=audit(1794831843.332:433): apparmor="ALLOWED" operation="open" ... no audit daemon the kernel log audit: type=1400 audit(1794831779.488:174): apparmor="DENIED" operation="open" ... dmesg journalctl -k the prefix differs, the record after it is the same 1400 is the number of the audit type AVC after installing auditd: no AppArmor line in dmesg, every record in audit.log
AppArmor hands each record to the kernel's audit system. With the audit daemon running the record goes to /var/log/audit/audit.log as a line starting type=AVC msg=audit. Without an audit daemon the kernel prints it to the kernel log, shown by dmesg and journalctl -k, as a line starting audit: type=1400 audit. The text after the prefix is the same. After installing auditd on the test machine, dmesg got no more AppArmor lines and all of them were in audit.log.

With the audit daemon running, records are in /var/log/audit/audit.log as type=AVC. Without it they are in the kernel log (dmesg, journalctl -k) as audit: type=1400. The text after the prefix is the same.

Where's the line? AppArmor hands every record to the kernel's audit system, and from there it goes one of two ways. If the audit daemon is running, the record lands in the audit log, as a line of type AVC. If no audit daemon is running, the kernel prints the record into its own log, and you see it with dmesg, or in the journal's kernel messages. The prefix differs; the text after it is the same. So know which case you're in. We installed the audit daemon on a test machine, and from that moment the kernel log got no AppArmor lines at all. They hadn't stopped. They'd moved.

Details and sources

Details

  • Logged by a Debian sid guest (kernel 7.3.4, a distribution kernel without Ubuntu's AppArmor patches); the field names and their order were checked against the kernel source the map describes. Without auditd the records were in dmesg and in journalctl -k: Nov 16 12:23:31 aamap kernel: audit: type=1400 audit(1794831811.744:217): apparmor="DENIED" ....
  • After apt-get install auditd on the same guest: auditctl -s printed enabled 1, rate_limit 0, backlog_limit 8192, lost 0; new records were in /var/log/audit/audit.log (type=AVC msg=audit(1794831843.332:433): apparmor="ALLOWED" operation="open" class="file" profile="demo-complain" ...) and the number of apparmor= lines in dmesg stayed 0.
  • The number in parentheses is the time in seconds since 1970 with milliseconds, a colon and a serial number. Two records with the same number belong to one event.
  • In audit.log, file records end fsuid=0 ouid=0FSUID="root" OUID="root": auditd appends the names it looked up after a separator character that a terminal does not show.
  • /var/log/syslog and /var/log/kern.log hold the kernel log too on systems that run a syslog daemon. On the Debian sid image neither exists; on the Ubuntu guests both exist and had the records.
  • The utilities read the first file that exists of /var/log/audit/audit.log, /var/log/syslog, /var/log/messages (logprof.conf:15).
  • Records of D-Bus mediation are written by the bus daemon, not by the kernel. They have type 1107 (USER_AVC) and carry the AppArmor text inside msg='...': see D-Bus mediation and the reference below.
  • In a container the kernel log is the host's: the records of a confined container process are on the host.
  • The kernel log is rate limited; the audit log was not in the test. See the scene "Complain mode, and the lines that never appear".

for AppArmor developers for security researchers

In the code

03 The first word: five kinds of record

APPARMOR="..." DENIEDrefused: the call failedapparmor="DENIED" operation="open" ... profile="demo" name="/etc/demo/secret.conf" ALLOWEDcomplain mode: would be refused, and went throughapparmor="ALLOWED" operation="open" ... profile="demo-complain" AUDITallowed by a rule marked auditapparmor="AUDIT" ... name="/etc/demo/audited.conf" ... requested_mask="r" KILLEDrefused, and the process got a signalapparmor="KILLED" operation="open" ... profile="demo-kill" STATUSpolicy changed: a profile was loaded, replaced or removedapparmor="STATUS" operation="profile_load" profile="unconfined" name="demo"
Five values of the first field, each with a real record. DENIED: refused, the call failed. ALLOWED: complain mode, the access would be refused and went through. AUDIT: allowed by a rule marked audit. KILLED: refused and the process was signalled. STATUS: policy changed, a profile was loaded, replaced or removed; the record has operation profile_load, profile unconfined and name demo.

apparmor= is DENIED (refused), ALLOWED (complain mode: would be refused), AUDIT (allowed by an audit rule), KILLED (refused and signalled) or STATUS (policy changed).

The first field tells you what kind of line you have. Denied: the profile refused, and the call failed. Allowed is the one that misleads people. It doesn't mean a rule allowed it. It means the profile is in complain mode: the access would have been refused, and went through. Audit is the opposite case: a rule did allow it, and the rule was marked audit, so you get a record. Killed: refused, and the process got a signal as well. And status isn't about a program's access at all: a profile was loaded, replaced or removed. Here it's the parser loading demo.

Details and sources

Details

  • All five lines were logged in one run. Logged by a Debian sid guest (kernel 7.3.4, a distribution kernel without Ubuntu's AppArmor patches); the field names and their order were checked against the kernel source the map describes. On the slide the middle of each record is replaced by ....
  • DENIED: apparmor="DENIED" operation="open" class="file" profile="demo" name="/etc/demo/secret.conf" pid=1323 comm="cat" requested_mask="r" denied_mask="r" fsuid=0 ouid=0; cat printed Permission denied.
  • ALLOWED: the profile demo-complain has flags=(complain) and no rule for the file: apparmor="ALLOWED" operation="open" class="file" profile="demo-complain" name="/etc/demo/secret.conf" pid=1468 comm="cat" requested_mask="r" denied_mask="r" fsuid=0 ouid=0; cat printed the file. The record still has denied_mask: it says what would have been denied.
  • AUDIT: the rule is audit /etc/demo/audited.conf r,: apparmor="AUDIT" operation="open" class="file" profile="demo" name="/etc/demo/audited.conf" pid=1474 comm="cat" requested_mask="r" fsuid=0 ouid=0. There is no denied_mask. A second AUDIT record with operation="getattr" followed.
  • KILLED: the profile demo-kill has flags=(kill): apparmor="KILLED" operation="open" class="file" profile="demo-kill" name="/etc/demo/secret.conf" pid=1494 comm="cat" requested_mask="r" denied_mask="r" fsuid=0 ouid=0; the shell reported Killed and exit status 137.
  • STATUS: apparmor="STATUS" operation="profile_load" profile="unconfined" name="demo" pid=1292 comm="apparmor_parser". Here profile= is the label of the process that changed policy and name= is the profile that was changed. Loading the same profile again logged operation="profile_replace" info="same as current profile, skipping"; removing it operation="profile_remove".
  • A load that the kernel refuses is a STATUS record with info= and error=: 300 random bytes written to .load logged apparmor="STATUS" info="invalid profile format" error=-93 profile="unconfined" pid=1317 comm="cat".
  • The kernel's table has three more words, HINT, ERROR and AUTO. None was produced in the run; AUTO is replaced before a record is written.
  • Known discrepancy: Kernel: an audit deny file rule in a complain-mode profile is allowed and logged ALLOWED.

for AppArmor developers for security researchers

In the code

04 Who was stopped, doing what, to what

THE COMMON FIELDS apparmor="DENIED" operation="open" class="file" profile="demo"name="/etc/demo/secret.conf" pid=1323 comm="cat"requested_mask="r" denied_mask="r" fsuid=0 ouid=0 profile="demo"the profile to edit comm="cat"command name, not a pathpid=1323the process operation="open"what the kernel was checkingclass="file"the kind of rule consulted name="/etc/demo/secret.conf"the object: here a path info="failed af match" error=-13 the kernel's reason, when it has one (here from a network record; -13 is EACCES)
The open record again, with six fields explained. profile demo: the profile that decided, the one to edit. comm cat: the command name, 15 characters, not a path. pid 1323: the process. operation open: what the kernel was checking. class file: the kind of rule consulted. name /etc/demo/secret.conf: the object, here a path. A network record also has info failed af match and error -13: the kernel's reason, when it has one.

profile is the profile to edit; comm is only the command name. operation and class say what was checked and by which kind of rule; name is the object. info and error give a reason when the kernel has one.

Now the fields that every record has. Start with profile. That's the profile that made the decision, and so the one to edit. Don't take it from comm: comm is only the command name, cut to fifteen characters, and for a script it's the interpreter. The pid is the process. Operation is what the kernel was checking, here an open, and class is the kind of rule that was consulted: file. Then name: the object. For a file, that's the path, exactly as AppArmor saw it. Some records carry two more fields, info and error. That's the kernel giving its reason, when it has one beyond no rule matching.

Details and sources

Details

  • Logged by a Debian sid guest (kernel 7.3.4, a distribution kernel without Ubuntu's AppArmor patches); the field names and their order were checked against the kernel source the map describes. On the slide the prefix audit: type=1400 audit(SECONDS:SERIAL): is left out and the record is broken over lines.
  • The kernel writes the common fields in this order: apparmor, operation, class, info and error (only when there is a reason), namespace (only when the profile is not in the root policy namespace), profile (or label when the subject is a stack of profiles), name, then pid and comm, then the fields of the class.
  • operation values seen in the run: open, mknod (creating a file), exec, link, chown, getattr, connect, create (a socket), capable, signal, ptrace, mount, umount, userns_create, change_onexec, profile_load, profile_replace, profile_remove. The complete list is in the reference below.
  • class values seen: file, cap, net, signal, ptrace, mount, namespace. change_onexec is logged with class="file".
  • comm is the task's command name, at most 15 characters. In the run a refused exec had comm="sh" (the caller) and name="/opt/demo/helper" (the program that was to run); Python scripts had comm="python3".
  • name is quoted. When it contains a space, a quote or a control character the kernel writes it as hexadecimal digits without quotes; libapparmor and the utilities decode it.
  • name is not always a full path. The mount command's look at a mount that is not attached yet logged operation="getattr" class="file" ... name="", and opening the POSIX message queue /demo logged operation="open" class="file" ... name="demo", both with info="Failed name lookup - disconnected path" error=-13: the object had no path from the task's root, so name is not a path a file rule can be written from. See The other profile flags for attach_disconnected.
  • The message queue record is a file record because this kernel has no message queue mediation: the queue is opened as a file on a mount that is attached nowhere. A profile with mqueue (create, open, read, write, delete) type=posix /demo-ok, got the same refusal for /demo-ok: apparmor="DENIED" operation="open" class="file" info="Failed name lookup - disconnected path" error=-13 profile="demo-mq" name="demo-ok" pid=1455 comm="python3" requested_mask="wrc" denied_mask="wrc" fsuid=0 ouid=0. The same was logged on Debian 13. What kernels with message queue mediation log instead is in Upstream and Ubuntu kernels.
  • Known discrepancy: a POSIX message queue on a kernel without mqueue mediation is refused as a file with a disconnected path.
  • info values seen: failed af match (no rule for the address family), failed mntpnt match, Failed name lookup - disconnected path, same as current profile, skipping, invalid profile format. error is a negative errno: -13 EACCES, -93 EPROTONOSUPPORT.
  • For a STATUS record profile= is the label of the process that changed policy and name= the profile that changed.
  • Known discrepancy: Kernel: the audit class name table has io_uring at index 18.

for AppArmor developers for security researchers

In the code

Read more

05 Which permission, and whose file

MASKS AND OWNERS apparmor="DENIED" operation="open" class="file" profile="demo" name="/srv/demo/root-owned.txt" pid=1336 comm="cat" requested_mask="r" denied_mask="r" fsuid=1000 ouid=0 requested_mask="r" what was asked for denied_mask="r" the part that was refused r w a x m k l the letters of a rule operation="mknod" ... name="/var/lib/demo/new" ... denied_mask="c" /var/lib/demo/new w, c (create) and d (delete) appear in records only: a rule says w fsuid=1000 the user who asked ouid=0 the file's owner owner /srv/demo/** r, they differ, so the owner rule does not apply operation="link" ... name="/var/lib/demo/link" ... denied_mask="l" ... target="/etc/demo/demo.conf" a link record adds target: the existing file
The rule owner /srv/demo/** r and a denial for /srv/demo/root-owned.txt read by another user: requested_mask r, denied_mask r, fsuid 1000, ouid 0. denied_mask is the part refused, in rule letters r w a x m k l; c and d are written w in a rule. fsuid is who asked, ouid is who owns the file; they differ and the rule says owner. A second record for touch on /var/lib/demo/new has requested_mask c, and the rule is /var/lib/demo/new w. A link record adds target, the existing file.

denied_mask is the refused part in rule letters; c and d are written w in a rule. fsuid is who asked and ouid who owns the file: when they differ, an owner rule does not apply.

For a file, two masks say which permission. The requested_mask is what was asked for, the denied_mask the part that was refused, in the letters you write in a rule. Two letters differ: c for create and d for delete. A rule has neither; you write w. So touch on a new file logs c, and the rule says w. Then fsuid and ouid: the user who asked, and the user who owns the file. Here they're one thousand and zero, and the rule says owner. That's the reason for this denial. A link record has one field more: target, the existing file.

Details and sources

Details

  • Logged by a Debian sid guest (kernel 7.3.4, a distribution kernel without Ubuntu's AppArmor patches); the field names and their order were checked against the kernel source the map describes. On the slide the prefix audit: type=1400 audit(SECONDS:SERIAL): is left out and the record is broken over lines.
  • The first record: the profile has owner /srv/demo/** r,; /srv/demo/root-owned.txt belongs to root with mode 644; it was read by the user tester (uid 1000) with runuser -u tester -- aa-exec -p demo -- cat /srv/demo/root-owned.txt. File modes would allow the read; the profile does not, because the rule applies only when the task's fsuid is the file's owner.
  • The letters: r read, w write, a append, x execute, m map executable, k lock, l link, and in records also c create and d delete. A check of file attributes (getattr) is logged as r; chmod, chown and setattr are logged as w.
  • For a denial the kernel reduces requested_mask to the permissions that were not allowed before it writes the record, so the two masks are normally the same. An AUDIT record has requested_mask only.
  • The second record in full: apparmor="DENIED" operation="mknod" class="file" profile="demo" name="/var/lib/demo/new" pid=1342 comm="touch" requested_mask="c" denied_mask="c" fsuid=0 ouid=0.
  • The link record in full: apparmor="DENIED" operation="link" class="file" profile="demo" name="/var/lib/demo/link" pid=1355 comm="ln" requested_mask="l" denied_mask="l" fsuid=0 ouid=0 target="/etc/demo/demo.conf". name is the new link, target the existing file; the rule is link /var/lib/demo/link -> /etc/demo/demo.conf, or l on the new name: see Link rules.
  • chown on a file is refused as a file operation before any capability is asked: apparmor="DENIED" operation="chown" class="file" profile="demo" name="/etc/demo/demo.conf" pid=1361 comm="chown" requested_mask="w" denied_mask="w" fsuid=0 ouid=0. The rule it maps to is a w file rule.
  • A unix socket in the filesystem is a file record with both letters: operation="connect" class="file" profile="demo" name="/run/demo.sock" ... requested_mask="wr" denied_mask="wr".
  • fsuid and ouid are numbers as seen from the initial user namespace. In audit.log the audit daemon appends FSUID="root" OUID="root" with the names.
  • target is also the profile of a transition: apparmor="DENIED" operation="change_onexec" class="file" profile="demo" name="demo-complain" pid=1439 comm="aa-exec" target="demo-complain".

for AppArmor developers for security researchers

In the code

06 Not a file: capabilities, sockets, signals, mounts

OTHER CLASSES capabilityoperation="capable" class="cap" ... capability=18 capname="sys_chroot"capability sys_chroot,capname is the word networkoperation="create" class="net" ... family="inet" sock_type="stream" protocol=0network inet stream,family, then sock_type signaloperation="signal" ... denied_mask="send" signal=hup peer="unconfined"signal (send) set=(hup) peer=unconfined,peer: the other profile mountoperation="mount" ... name="/mnt/demo/" ... fstype="tmpfs" srcname="none"mount fstype=tmpfs none -> /mnt/demo/,name is the mount point the record's values are the rule's words

Other classes carry their own fields: capname; family and sock_type; signal and peer; fstype, srcname and name. Each value is a word of the rule.

When class isn't file, there's no path to look for. Each class brings fields of its own, and they still spell the rule. A capability record has capname, and that's the word for the rule. A network record has family and sock_type: inet and stream give you network inet stream. A signal record names the signal, and peer: the profile of the process on the other end. Unconfined, here. A mount record has the filesystem type and the source, and name is the mount point. The decks on each kind of rule show their record in full. The pattern is always the same: the record's values are the rule's words.

Details and sources

Details

  • Logged by a Debian sid guest (kernel 7.3.4, a distribution kernel without Ubuntu's AppArmor patches); the field names and their order were checked against the kernel source the map describes. On the slide the middle of each record is replaced by .... The four rules were compiled by the tree's parser without a message.
  • Capability, in full: apparmor="DENIED" operation="capable" class="cap" profile="demo" pid=1367 comm="chroot" capability=18 capname="sys_chroot" (from chroot / true, which failed with Operation not permitted). The two spaces before capname are in the record.
  • Network: apparmor="DENIED" operation="create" class="net" info="failed af match" error=-13 profile="demo" pid=1379 comm="python3" family="inet" sock_type="stream" protocol=0 requested="create" denied="create". Network records use requested and denied with words (create, connect, send, receive, bind, listen, accept, getattr, setattr, getopt, setopt, shutdown), not the _mask fields.
  • Unix sockets add addr, peer_addr and peer: apparmor="DENIED" operation="connect" class="net" profile="demo" pid=1387 comm="python3" family="unix" sock_type="stream" protocol=0 requested="send receive connect" denied="send connect" addr=none peer_addr="@demo" peer="unconfined". The rule: unix (connect, send) type=stream peer=(addr=@demo label=unconfined),. See Unix socket rules.
  • Signal: apparmor="DENIED" operation="signal" class="signal" profile="demo" pid=1394 comm="sh" requested_mask="send" denied_mask="send" signal=hup peer="unconfined". The receiving side is checked against its own profile and logs its own record with receive.
  • ptrace: reading /proc/1/environ logged apparmor="DENIED" operation="ptrace" class="ptrace" profile="demo" pid=1401 comm="cat" requested_mask="read" denied_mask="read" peer="unconfined"; the rule is ptrace (read) peer=unconfined,.
  • Mount: apparmor="DENIED" operation="mount" class="mount" info="failed mntpnt match" error=-13 profile="demo" name="/mnt/demo/" pid=1407 comm="mount" fstype="tmpfs" srcname="none". This record is from LIBMOUNT_FORCE_MOUNT2=always mount -t tmpfs none /mnt/demo, which makes the mount command use the mount(2) system call. With the command's default system calls the record was operation="mount" class="mount" info="failed mntpnt match" error=-13 profile="demo" name="/mnt/demo/" pid=1413 comm="mount" flags="rw, move", without fstype and srcname.
  • Known discrepancy: Kernel: mount_setattr(2), fsconfig reconfigure and fsopen/fsmount are not checked as remounts or new mounts.
  • Unmount: apparmor="DENIED" operation="umount" class="mount" profile="demo" name="/mnt/demo/" pid=1420 comm="umount"; the rule is umount /mnt/demo/,.
  • User namespace: apparmor="DENIED" operation="userns_create" class="namespace" profile="demo" pid=1427 comm="unshare" requested="userns_create" denied="userns_create" execpath="/usr/bin/unshare"; the rule is userns,.
  • Records that were not produced in the run (setrlimit, io_uring, pivotroot) are in the decks of their rules.

for AppArmor developers for security researchers

In the code

07 Complain mode, and the lines that never appear

ALLOWED, AND MISSING apparmor="ALLOWED" operation="open" class="file" profile="demo-complain" name="/etc/demo/secret.conf" ... requested_mask="r" denied_mask="r" the read went through deny /etc/demo/quiet.conf r, refused no record 20 refused chroot() calls in a second one record 201 denials, no audit daemon 11 lines in the kernel log kauditd_printk_skb: 339 callbacks suppressed 200 denials, audit daemon running 200 lines in the audit log building a profile from the log? run the audit daemon no line does not mean the access was allowed
A complain mode record: apparmor ALLOWED, profile demo-complain, the read went through. Then three refusals without a line. The rule deny /etc/demo/quiet.conf r: refused, no record. 20 refused chroot calls in one second: one record. 201 denials with no audit daemon: 11 lines in the kernel log and the note kauditd_printk_skb: 339 callbacks suppressed. With the audit daemon running, 200 denials gave 200 lines in the audit log.

Complain mode logs ALLOWED and lets the access through. No line is written for a plain deny, for a capability refused again within a second, or for records beyond the kernel log's rate limit: 11 lines for 201 denials in the test.

The demo-complain profile is in complain mode. The same read goes through, and the line says allowed, with the same fields. That's how you learn what a program needs without breaking it. Now the lines that never appear. A plain deny rule refuses quietly: no record. A capability refused twice within a second is logged once. When no audit daemon runs, the kernel log has a rate limit. We ran two hundred and one denials, and eleven lines came out, with a note that the rest was suppressed. With the audit daemon running, every one was there. So if you're building a profile from the log, run the audit daemon.

Details and sources

Details

  • Logged by a Debian sid guest (kernel 7.3.4, a distribution kernel without Ubuntu's AppArmor patches); the field names and their order were checked against the kernel source the map describes.
  • Complain mode: the profile demo-complain has flags=(complain); cat /etc/demo/secret.conf under it printed the file and logged apparmor="ALLOWED" operation="open" class="file" profile="demo-complain" name="/etc/demo/secret.conf" pid=1468 comm="cat" requested_mask="r" denied_mask="r" fsuid=0 ouid=0. A plain deny rule still refuses in complain mode: see Profile mode flags.
  • Known discrepancy: Kernel: an audit deny file rule in a complain-mode profile is allowed and logged ALLOWED.
  • Plain deny: with deny /etc/demo/quiet.conf r, in the profile, cat printed Permission denied and the kernel log got no line. audit deny keeps the record: see The audit qualifier.
  • The module's audit mode can bring quieted records back: after echo -n noquiet > /sys/module/apparmor/parameters/audit the same read logged apparmor="DENIED" operation="open" class="file" profile="demo" name="/etc/demo/quiet.conf" pid=1487 comm="cat" requested_mask="r" denied_mask="r" fsuid=0 ouid=0. The values are normal (the default), quiet_denied, quiet.allowed, quiet, noquiet and all; quiet_denied and quiet remove denials, and quiet.allowed removes the ALLOWED records of complain mode (only noquiet was tried; the rest is from the source). A value other than normal overrides every profile's own audit flag.
  • Capabilities: a Python process called chroot("/") 20 times; all 20 failed and one record was logged (capname="sys_chroot"). The duplicate filter is per capability and lasts one second. File operations have no such filter (from the source; the recorded run refused one chown and logged one record).
  • The rate limit: kernel.printk_ratelimit = 5 and kernel.printk_ratelimit_burst = 10 are the guest's defaults. A shell loop ran 200 refused reads, and one more 7 seconds later. dmesg then had 11 DENIED lines and the line kauditd_printk_skb: 339 callbacks suppressed. Why the counter says 339 for 190 missing lines was not determined.
  • With auditd installed and running (auditctl -s: rate_limit 0, backlog_limit 8192), the same loop of 200 added 200 lines to /var/log/audit/audit.log, and lost stayed 0.
  • sysctl -w kernel.printk_ratelimit=0 switches the kernel log's limit off; the run did that for every other test.
  • The audit daemon has limits of its own (auditctl -r for a rate limit, the backlog limit); they were at their defaults here.

for AppArmor developers for security researchers

In the code

08 From the record to one new rule

RECORD TO RULE apparmor="DENIED" operation="open" class="file" profile="demo" name="/etc/demo/secret.conf" ... requested_mask="r" denied_mask="r" profile demo { /etc/demo/demo.conf r, /etc/demo/secret.conf r, ... } one new line: name, and the letter of denied_mask keep the path as narrow as the program needs $ apparmor_parser -r /etc/apparmor.d/demo apparmor="STATUS" operation="profile_replace" ... name="demo" $ aa-exec -p demo -- cat /etc/demo/secret.conf secret no new record then expect the next denial aa-logprof does this loop for you aa-notify shows denials as they arrive
The denial record with three fields marked: profile demo, name /etc/demo/secret.conf, denied_mask r. The profile demo gets one new line, /etc/demo/secret.conf r. The command apparmor_parser -r /etc/apparmor.d/demo reloads it and the kernel logs a STATUS record with operation profile_replace. Running cat under the profile again prints the file and no record is logged. aa-logprof does this loop for you and aa-notify shows denials as they arrive.

profile names the profile to edit; name and denied_mask are the rule. Reload with apparmor_parser -r, run again, and expect the next denial.

So, from the record to a rule. First ask whether demo should have this file at all; a denial can be the profile doing its job. Say it should. The profile field says which profile to edit. The name and the denied_mask are the rule: the path, and r. Keep the path as narrow as the program needs. Then reload the profile with the parser's -r option; a status record confirms the replacement. Run demo again. It works, and there's no new record. Expect more rounds: a program usually stops at its first denial. The aa-logprof tool does this loop for you, and aa-notify shows denials as they arrive.

Details and sources

Details

  • Logged by a Debian sid guest (kernel 7.3.4, a distribution kernel without Ubuntu's AppArmor patches); the field names and their order were checked against the kernel source the map describes. The profile with the added line is the one of the run with /etc/demo/secret.conf r, after the rule for demo.conf. apparmor_parser -r exited with status 0; aa-exec -p demo -- cat /etc/demo/secret.conf printed secret; the kernel log got no new line.
  • In the guest the profile was a file in a scratch directory and was loaded by its path. The slide writes /etc/apparmor.d/demo, where a profile normally lives; the STATUS line on the slide is shortened with .... The run did not keep the kernel log of this reload. The reloads it did keep replaced a profile with an identical one and logged apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="demo" pid=1299 comm="apparmor_parser"; for a profile that changed, the kernel writes the same record without info (from the source, not run).
  • The steps, with the field that feeds each: profile is the profile to edit (aa-status lists what is loaded; the file is usually under /etc/apparmor.d/). class and operation say which kind of rule. name and denied_mask, or the class's own fields, are the rule's words. fsuid and ouid say whether owner fits.
  • Write c and d as w. Generalise the path with care: /etc/demo/*.conf r, covers the directory's configuration files; /etc/demo/** rw, would give away much more than the record asked for.
  • -r replaces a loaded profile. Running processes keep the profile and get the new rules at once.
  • Where the profile includes a site file (include if exists <local/NAME>), put a local addition there instead of editing the shipped file: see include if exists and the Denials quick guide.
  • A record with info="Failed name lookup - disconnected path" is the exception to "name and denied_mask are the rule": its name is empty or has no leading slash, so it is not a path that a file rule can name. The scene "Who was stopped, doing what, to what" has two such records. Whether attach_disconnected and a file rule make such an access work was not tried.
  • Not every record should become a rule. A program that probes for things it does not need works without them; a deny rule for such a path also silences its record.
  • aa-logprof and aa-notify read the same records. Neither was installed on the guest's image; they are separate packages.

09 From the log to the tools and the rules

ROADS apparmor="DENIED" operation="open" ... profile="demo" ut-logprofut-notify the tools that read the log syn-auditsyn-allow-denysyn-flags-modes why a line is or is not there syn-access-modessyn-owner the masks and the uids adm-featuresupstream-and-ubuntudbus-mediationk-audit rules that load and are not enforced the kernel's side of the record
A denial record in the centre. Roads to the tools ut-logprof and ut-notify; to syn-audit, syn-allow-deny and syn-flags-modes, which decide whether a line is written; to syn-access-modes and syn-owner for the masks and the uids; and to k-audit, adm-features, upstream-and-ubuntu and dbus-mediation.

First word, then who, what and to what, then the permission. Roads: the tools that read the log, the qualifiers and flags that decide whether a line is written, and the kernel side.

That's a denial, from left to right. The first word says the kind of record. Profile, operation and name say who was stopped, doing what, to what. The masks, or the class's own fields, give the rule's words. Silence proves nothing: a plain deny, a repeated capability or the rate limit leave no line. From here: the tools that go through the log, aa-logprof and aa-notify. The decks on audit, deny and the mode flags explain why a line is or isn't there. The deck on kernel features covers rules that load and aren't enforced. And the kernel's side of the record is in how a denial is logged.

Details and sources

Read more

Basis of the statements on this page: "logged" means the line was written by a running kernel, a Debian sid guest with kernel 7.3.4 (a distribution kernel without Ubuntu's AppArmor patches), under profiles compiled by the parser of the AppArmor commit 836052d21. "source" means the kernel source the map describes was read and the record was not produced. Records are quoted without the prefix audit: type=1400 audit(SECONDS:SERIAL):. What other kernels add to these records is in Upstream and Ubuntu kernels.

Where records are

SituationWhereLine startsCommands
the audit daemon (auditd) is running/var/log/audit/audit.logtype=AVC msg=audit(1794831843.332:433):grep apparmor= /var/log/audit/audit.log
no audit daemonthe kernel logaudit: type=1400 audit(1794831779.488:174):dmesg, journalctl -k
a syslog daemon is installedalso /var/log/syslog, /var/log/kern.logthe kernel log's line with a time stampgrep apparmor= /var/log/kern.log
D-Bus mediationthe same two placestype=1107 in the kernel log (USER_AVC), the AppArmor text inside msg='...'

Logged: both forms, on one guest before and after installing auditd; after it, dmesg received no AppArmor line. The utilities read the first existing file of /var/log/audit/audit.log, /var/log/syslog, /var/log/messages (logprof.conf:15).

Record types

apparmor=MeaningCall resultBasis
DENIEDthe profile refused the accessfails, usually EACCESlogged
ALLOWEDcomplain mode: the access would be refusedsucceedslogged
AUDITa rule with audit (or the audit mode all) allowed the accesssucceedslogged
KILLEDkill mode or a kill rule: refusedfails, and the profile's signal is sent (the test process was killed, exit status 137)logged
STATUSa profile was loaded, replaced or removed, or a load failedlogged
HINT, ERRORin the kernel's tablesource
AUTOinternal; replaced before a record is writtensource

Common fields

In the order the kernel writes them.

FieldValueSaysBecomes
apparmorone of the words abovethe kind of record
operationa wordwhat was checkedwith class, the kind of rule
classfile, cap, net, rlimits, domain, mount, ptrace, signal, namespace, io_uring, dbus, ...the rule class consultedthe rule's keyword
infotexta reason
errornegative errno, only with info
namespacename, only outside the root policy namespacewhere the profile lives
profilenamethe profile that decided; in STATUS the label of the process changing policythe profile to edit
labellabel, instead of profile for a stack
namequoted, or hexadecimal without quotes when it has a space, a quote or a control characterthe object: a path, a mount point, a target profile; in STATUS the profile changedthe path of the rule
pidnumberthe process
commat most 15 charactersthe command name

Operations in the source: open, exec, mknod, mkdir, rmdir, unlink, truncate, link, symlink, rename_src, rename_dest, chmod, chown, getattr, file_receive, file_perm, file_lock, file_mmap, file_mprotect, file_inherit; mount, umount, pivotroot; create, post_create, bind, connect, listen, accept, sendmsg, recvmsg, getsockname, getpeername, getsockopt, setsockopt, socket_shutdown; ptrace, signal; capable, sysctl; change_hat, change_profile, change_onexec, stack, stack_onexec; setprocattr, setrlimit; userns_create; uring_override, uring_sqpoll; profile_load, profile_replace, profile_remove.

Fields by class

ClassFieldsRuleLogged example
filerequested_mask, denied_mask, fsuid, ouid, targetNAME LETTERS, (c, d as w); owner when the uids match; link NAME -> TARGET,operation="open" class="file" profile="demo" name="/srv/demo/root-owned.txt" pid=1336 comm="cat" requested_mask="r" denied_mask="r" fsuid=1000 ouid=0
capcapability, capnamecapability CAPNAME,operation="capable" class="cap" profile="demo" pid=1367 comm="chroot" capability=18 capname="sys_chroot"
netfamily, sock_type, protocol, requested, deniednetwork FAMILY SOCK_TYPE,operation="create" class="net" info="failed af match" error=-13 profile="demo" pid=1379 comm="python3" family="inet" sock_type="stream" protocol=0 requested="create" denied="create"
net, unix socketsalso addr, peer_addr, peerunix (DENIED) type=SOCK_TYPE peer=(addr=PEER_ADDR label=PEER),operation="connect" class="net" profile="demo" pid=1387 comm="python3" family="unix" sock_type="stream" protocol=0 requested="send receive connect" denied="send connect" addr=none peer_addr="@demo" peer="unconfined"
signalrequested_mask, denied_mask, signal, peersignal (DENIED) set=(SIGNAL) peer=PEER,operation="signal" class="signal" profile="demo" pid=1394 comm="sh" requested_mask="send" denied_mask="send" signal=hup peer="unconfined"
ptracerequested_mask, denied_mask, peerptrace (DENIED) peer=PEER,operation="ptrace" class="ptrace" profile="demo" pid=1401 comm="cat" requested_mask="read" denied_mask="read" peer="unconfined"
mountfstype, srcname, trans, flags, optionsmount fstype=FSTYPE options=(FLAGS) SRCNAME -> NAME,; umount NAME,operation="mount" class="mount" info="failed mntpnt match" error=-13 profile="demo" name="/mnt/demo/" pid=1407 comm="mount" fstype="tmpfs" srcname="none"
namespacerequested, denied, execpathuserns,operation="userns_create" class="namespace" profile="demo" pid=1427 comm="unshare" requested="userns_create" denied="userns_create" execpath="/usr/bin/unshare"
rlimitsrlimit, value, peerset rlimit RLIMIT <= VALUE,source
io_uringrequested, denied, tcontextio_uring (...) label=TCONTEXT,source
transitionstargetchange_profile -> TARGET,, an exec ruleoperation="change_onexec" class="file" profile="demo" name="demo-complain" pid=1439 comm="aa-exec" target="demo-complain"
policy load errorsns, name, offsetapparmor="STATUS" info="invalid profile format" error=-93 profile="unconfined" pid=1317 comm="cat"
D-Bus (bus daemon)bus, path, interface, member, mask, name, label, peer_labeldbus (MASK) bus=BUS path=PATH interface=INTERFACE member=MEMBER peer=(label=PEER_LABEL),type=1107 ... msg='apparmor="DENIED" operation="dbus_method_call" bus="system" path="/" interface="org.freedesktop.DBus" member="ListNames" mask="send" name="org.freedesktop.DBus" pid=1505 label="demo-dbus" peer_label="unconfined"

The mount row is the record of the mount(2) system call. The mount command's default system calls logged flags="rw, move" without fstype and srcname. Known discrepancies: Kernel: mount_setattr(2), fsconfig reconfigure and fsopen/fsmount are not checked as remounts or new mounts; Kernel: mount audit records have no word for lazytime, and MS_REC shows only with bind or a propagation flag; Kernel: signal=unknown(%d) is logged without a leading space; Kernel: changeprofile &name is audited as operation="change_profile".

Mask letters

LetterPermissionIn a rule
rread; also getattrr
wwrite; also chmod, chown, setattrw
aappenda
ccreatew
ddeletew
xexecutean exec mode such as ix or Px
mmap executablem
klockk
llinkl

info values

Logged: failed af match, failed mntpnt match, Failed name lookup - disconnected path, same as current profile, skipping, invalid profile format. From the source, among others: failed type match, failed protocol match, profile transition not found, label not found, no new privs, ix fallback, ux fallback, target restricted, link not subset of target, hat not found, conflicting profile attachments.

A record with info="Failed name lookup - disconnected path" has a name that is empty or relative, and a file rule cannot be written from it. Logged: name="" for the mount command's look at a detached mount, and name="demo", requested_mask="wrc" for opening the POSIX message queue /demo, also under a profile that has an mqueue rule for the queue. Known discrepancy: a POSIX message queue on a kernel without mqueue mediation is refused as a file with a disconnected path.

What is not logged

CauseEffectBasis
a plain deny rulerefused, no record; audit deny logslogged
/sys/module/apparmor/parameters/auditnormal (default), quiet_denied and quiet remove denials, quiet.allowed removes the ALLOWED records of complain mode, noquiet logs quieted denials, all logs every access; a value other than normal overrides the profiles' own flagnoquiet logged; the rest source
the capability filterthe same capability for the same credentials: one record per secondlogged: 20 calls, 1 record
the kernel log's rate limit (no audit daemon)kernel.printk_ratelimit 5 seconds, burst 10logged: 201 denials, 11 lines
the audit daemon's own limitsauditctl -s shows rate_limit, backlog_limit, lostlogged with defaults: 200 of 200

From a record to a rule

1. Decide whether the access is wanted. 2. profile: the profile to edit. 3. class and operation: the kind of rule. 4. name and denied_mask, or the fields of the class: the words of the rule. 5. fsuid and ouid: whether owner fits. 6. apparmor_parser -r FILE; a STATUS record with operation="profile_replace" follows. 7. Run the program again and read the next record.

for AppArmor developers for security researchers

In the code

Roads from here

START HERE

FILE ACCESS

PROFILE HEAD

QUALIFIERS

RUNNING THE PARSER

TOOLS · utils/

  • aa-logprof — the tool that reads these records and proposes rules
  • aa-notify (no slides yet) — the tool that shows denials as they happen

LIBAPPARMOR

IN THE KERNEL