The binary policy format

binary layout for AppArmor developers for security researchers

A compiled profile is a stream of tagged elements - names and integers and strings and blobs and structs and arrays. Each profile and child and hat is one unit that starts with a version word. The three DFAs travel as aligned blobs and take most of the bytes. The stream may be compressed with zstd and the cache stores the same bytes. The kernel reads the fields in a fixed order and verifies the tables.

On the map: the dfa engine

▶ Watch this deck — 10 scenes, ~8 min

Worth seeing first

Part of journeys

01 5009 bytes for three rules

THE BINARY POLICY $ apparmor_parser -Q -S -K demo3.sd | xxd | head -3 5009 bytes 00000000: 0408 0076 6572 7369 6f6e 0002 0720 2000 ...version... .00000010: 0408 0070 726f 6669 6c65 0007 0505 0064 ...profile.....d00000020: 656d 6f00 0406 0061 6164 6661 0006 4605 emo....aadfa..F. version profile demo aadfa no profile text in here: names, numbers and tables the kernel one write to .load or .replace
The first three lines of a hex dump of the binary policy of the demo profile show the words version, profile, demo and aadfa; the whole stream is 5009 bytes and goes to the kernel in one write

apparmor_parser -S writes the binary policy to stdout: 5009 bytes for the three-rule example, starting with the element version.

Everything the other decks of this district built ends up as one run of bytes. The capital S option makes the parser print it instead of loading it. For demo's three rules that's five thousand and nine bytes. Look at the first lines of a hex dump and you can already make out some of it: the word version, the word profile, the name demo, and then something called aadfa. There's no trace of the profile text, no paths, no globs. It's a stream of named, typed values, and three of those values are the packed tables of the three DFAs. This deck goes through it in order, up to the write that hands it to the kernel.

Details and sources

Details

  • The command is apparmor_parser -Q -S -K demo3.sd, run with the parser built from the AppArmor tree and a feature set that has the policy entries the kernel exports; nothing was loaded into a kernel. -S writes the binary policy to standard output and loads nothing; -K skips the cache.
  • The profile (file demo3.sd):
abi <abi/5.0>,
include <tunables/global>
profile demo /usr/bin/demo {
  /etc/demo/** r,
  audit deny @{HOME}/.ssh/** rw,
  /usr/bin/[a-z]* ix,
}

Read more

02 One element: a name, a type, a value

sd_write_* 04080076657273696f6e000207202000 name 04 = NAME 08 00 = length 8, little-endian "version\0" value 02 = U32 07 20 20 00 = 0x00202007 0x202007 kernel ABI 7bits 0-9 parser ABI 2bits 12-19 policy version 2bits 20-27 force complain 0bit 11 0 U81 U162 U323 U644 NAME5 STRING6 BLOB 7 STRUCT8 STRUCTEND11 ARRAY12 ARRAYEND known discrepancy: with the tree's abi/5.0 as kernel features the word is 05 00 00 00
The first 16 bytes of the stream: a name element with code 4, length 8 and the text version, then a value element with code 2 for U32 and the little-endian number 0x00202007, which holds kernel ABI 7, parser ABI 2 and policy version 2; and the list of element codes

Element codes (parser/parser_interface.c:162-177): 0 U8, 1 U16, 2 U32, 3 U64, 4 NAME, 5 STRING, 6 BLOB, 7 STRUCT, 8 STRUCTEND, 11 ARRAY, 12 ARRAYEND. Stream integers are little-endian. The first element is version.

A value is a type byte followed by data. In front of it there may be a name, which is itself an element: code four, a sixteen bit length, and the characters with a closing NUL. So the first eleven bytes of the stream say: name, version. The next five say u32, and give the number, low byte first. The number is the interface version: kernel ABI seven in the low bits, and above it the parser's ABI and the policy version. The other codes are few: integers of four widths, strings, blobs with a thirty-two bit length, and brackets for structs and arrays. Compiled against the tree's own feature file, the word comes out as plain five.

Details and sources

Details

  • The codes are enum class sd_code (parser_interface.c:162-177). The forms (parser_interface.c:197-331): a name is 04, a 16-bit length that includes the NUL, and the bytes (parser_interface.c:252-261); a string is 05 with the same layout (parser_interface.c:286-297); a blob is 06, a 32-bit length and the bytes (parser_interface.c:263-269); an array is 0b, a 16-bit count, the elements and 0c (parser_interface.c:310-320); a struct is 07, the elements and 08 (parser_interface.c:299-308).
  • Codes 9 LIST, 10 LISTEND and 13 OFFSET are defined and not written. The kernel's enum aa_code has the same values up to ARRAYEND and no OFFSET (policy_unpack.h:63).
  • The version word is built by ENCODE_VERSION (parser.h:315-332, parser_common.c:61-63): bits 0-9 are the kernel ABI; when that is above 5, bit 11 is force complain, bits 12-19 the parser ABI (2) and bits 20-27 the policy version (2). kernel_abi_version is 7 or 6 when the features have policy/versions/v7 or v6, and 5 otherwise (parser_main.c:1763-1766).
  • Kernel readers: aa_unpack_nameX, aa_unpack_u32, aa_unpack_array, aa_unpack_blob, aa_unpack_str (policy_unpack.c:242, policy_unpack.c:291); every bounds check goes through aa_inbounds (policy_unpack.c:180).
  • verify_header (policy_unpack.c:1442) refuses a kernel ABI below 5 or above 9 with "unsupported interface version" (-EPROTONOSUPPORT). The force complain bit is FORCE_COMPLAIN_FLAG, 0x800 (policy_unpack.c:1224, policy_compat.h:17).
  • Kernel statements are read from the kernel source and were not run.
  • Known discrepancy: aa_features_supports(features, "policy/versions/v7") is false for a feature set that has policy/notify_versions, because the lookup matches versions against the end of notify_versions (features.c:361-444). The tree's abi/5.0 file has that entry, so with it the parser writes version 5, without policy version, parser ABI or force complain bit, and leaves the file class marker out of policydb. This was confirmed by running the tree's parser; the kernel's own feature directory has no notify_versions.

03 The profile struct, field by field

sd_serialize_profile / unpack_profile 001cstring "demo"the profile name 0024aadfa blob, then u32 13xmatch and xmatch_len 057dflags { 0, 0, 0 }flag bits, mode (0 = enforce), audit 05974 x u32, caps64 { 4 x u32 }capabilities 05cbpolicydb { perms, aadfa }unpack_pdb 0b29perms, permsv 3, aadfaunpack_pdb: the file DFA, at the top level absent here: disconnected kill path_flags xattrs rlimits xtable the kernel asks for the name, does not find it, steps back both sides must agree on the order
The fields of the profile struct of the example in stream order, each with its offset and the kernel side meaning: the name, the xmatch blob and its length, the flags struct, the capability words, the policydb struct and the file permission table with its DFA; optional fields that are absent are skipped by name

sd_serialize_profile (parser/parser_interface.c:480-604) writes name, xmatch, flags, capabilities, optional structs, policydb, then the file permission table and DFA. unpack_profile (linux:security/apparmor/policy_unpack.c:1121-1432) reads them in the same order.

Inside the profile struct the fields come in a fixed order. First the name, as a bare string. Then the attachment DFA, as a blob called aadfa, followed by the xmatch length: thirteen, the length of demo's path. Then a small struct of flags: flag bits, the mode, zero for enforce, and audit. Then eight words of capabilities. Then a struct called policydb, with its permission table and its DFA. And last, at the top level again, the file rules: a permission table, a version marker, and the third blob. The kernel asks for each optional field by name, and if the next element has another name, it steps back and moves on.

Details and sources

Details

  • void sd_serialize_profile(std::ostringstream &buf, Profile *profile, int flattened) (parser_interface.c:480-604); the reader is unpack_profile (policy_unpack.c:1121). All fields, in order (writer lines in parser_interface.c, reader lines in policy_unpack.c):
#FieldWritten whenWriterReader
1string: name (parent//child when flattened)always488-4931140-1158
2xmatch: blob aadfa, then u32 xmatch_lenxmatch exists496-5011175-1196
3string disconnectedflag attach_disconnected.path=504-5071199
4string disconnected_ipcset, and feature domain/disconnected.ipc509-512not read
5u32 killset, and feature domain/kill.signal514-5171203-1207
6u32 errorset, and feature policy/profile/error519-522not read
7struct flags: flag bits, mode, auditalways524-5291209-1244
8u32 path_flagsany path flag530-5451247-1252
9four u32: capability allow, audit, quiet, 0always549-5531255-1262
10struct caps64: the high halvesalways555-5601265-1277
11array identityfeature domain/identity562-563, 333-344not read
12struct xattrs: array of namesxattrs=565, 458-4781290, 591-620
13struct rlimits: mask, array of 16 u64any rlimit567, 398-4111295, 667-701
14array net_allowed_afold network encoding only570-584not read
15struct policydb: perms, blob aadfa, xtablepolicydb DFA exists586-5951305-1331
16perms, permsv, blob aadfa (file DFA)file DFA exists597-5981333-1352
17struct xtablefile DFA exists and a named exec target599-6021075 (in unpack_pdb)
18struct endalways6031407
  • The kernel also takes fields this parser does not write: rename, attach (policy_unpack.c:1169), capsx (policy_unpack.c:1280), secmark (policy_unpack.c:1300), tags, start and dfa_start (policy_unpack.c:1026), and a data struct (policy_unpack.c:1354).
  • Fields 4, 6, 11 and 14 are written only for a feature set that has the entry, and the kernel at this commit does not read them. A field the kernel does not know makes the next mandatory read fail ("profile missing flags").
  • The flags struct: flag bit 1 is a hat (PACKED_FLAG_HAT); mode 0 enforce, 1 complain, 2 kill, 3 unconfined, 4 user (policy_unpack.h:32).
  • Capability words: quiet is deny & quiet, and audit is limited to allowed capabilities (parser_interface.c:549-558).
  • Kernel error strings, in the info field of the audit record: "bad xmatch", "missing xmatch len", "profile missing flags", "failed to unpack profile flags", "failed to unpack profile capabilities", "failed to unpack policydb", "failed to unpack end of profile" (policy_unpack.c:1126).
  • Optional fields work through aa_unpack_nameX, which steps back when the name differs (policy_unpack.c:242).
  • Kernel statements are read from the kernel source and were not run.
  • Known discrepancies, by title: "attach_disconnected.ipc is dropped without a warning and its bit is written anyway"; "mediate_deleted is packed with the bit the kernel calls PATH_DELEGATE_DELETED".

04 Where the bytes go

5009 BYTES 0000 1391 xmatch1344 policydb1168 file1528 56: version, name 288: flags, caps, 2 perms rows 624: 8 perms rows, permsv 4040 of 5009 bytes: the three table sets next 568check 568 the file table set, 1528 bytes header 24, accept 104, accept2 104, base 104, default 56 277 slots for 49 transitions: a DFA is never smaller than 256 slots in next and in check
The 5009 bytes drawn to scale: the table sets of xmatch, policydb and the file DFA take 1344, 1168 and 1528 bytes, four fifths of the stream; the strips between them are the profile fields and permission tables; inside the file table set, next and check take 1136 of 1528 bytes

Of 5009 bytes, 4040 are the table sets of the three DFAs (1344, 1168, 1528). In the file DFA's 1528 bytes, next and check take 1136.

Let's see where five thousand bytes go. Draw the stream to scale. The attachment DFA's tables: thirteen hundred and forty-four bytes, for fifteen states. policydb: eleven hundred and sixty-eight, for five states. The file DFA: fifteen hundred and twenty-eight, for twenty-two. Together that's four thousand and forty bytes, four fifths of everything. The thin strips between them are the actual profile fields and the two permission tables. And inside each table set the picture repeats: next and check dominate, because of the rule from the tables deck that the arrays reach two hundred and fifty-six slots past the highest base. So a profile with almost no rules still weighs a few kilobytes.

Details and sources

Details

  • The elements of the 5009-byte stream (offsets hexadecimal; decoded from the -S output made with a feature set that has the kernel's policy entries):
OffsetElementValue
0000name version, u320x202007
0010name profile, struct
001cstringdemo
0024name aadfa, blobsize 1350 = 6 pad + table set of 1344 at 0038 (xmatch, 15 states)
0578u32xmatch_len 13
057dname flags, struct of 3 u320, 0 (enforce), 0
05974 u32capability allow, audit, quiet, 0: all 0
05abname caps64, struct of 4 u32all 0
05cbname policydb, struct
05d8name perms, struct: version 1, array of 2 rowsallow 0x8000, 0x8004
0683name aadfa, blobsize 1175 = 7 pad + table set of 1168 at 0698 (5 states)
0b28struct end (policydb)
0b29name perms, struct: version 1, array of 8 rowsallow 0x200 four times, 0x244 twice, 0x10201 twice
0d78name permsv, u323
0d87name aadfa, blobsize 1531 = 3 pad + table set of 1528 at 0d98 (22 states)
1390struct end (profile)total 0x1391 = 5009
  • Inside the file table set: header at 0d98, accept 0db0, accept2 0e18, base 0e80, default 0ee8, next 0f20, check 1158.
  • A permission table row is 14 u32 of 5 bytes each: 140 bytes for 2 rows, 560 for 8.
  • For comparison (tree's abi/5.0 feature set): the one-rule profile of 4529 bytes has DFAs of 15, 5 and 13 states and 3840 bytes of table sets; a profile with three abstractions is 46041 bytes for 1695 states in the file DFA.
  • Why the minimum: the arrays extend 255 slots past the highest base (chfa.cc:269-276), which the kernel requires (match.c:188).
  • Kernel statements are read from the kernel source and were not run.

Read more

05 A blob with padding in front

sd_write_aligned_blob 0406006161646661000646050000 name "aadfa" BLOB size 1350 000000000000 pad 6 1b5e783d table set at 0x38 = 56 = 7 x 8 unpack_dfa: skip size & 7 bytes works because a table set is a multiple of 8 long xmatch size 1350 pad 6 table set at 0038 policydb size 1175 pad 7 table set at 0698 file size 1531 pad 3 table set at 0d98 46 05 00 00 little-endian: the stream 00 00 05 40 big-endian: inside the table set (1344)
The bytes at offset 0x24: the name aadfa, the blob code 6, the little-endian size 1350, six zero bytes of padding and then the table set magic at offset 0x38, a multiple of 8; the kernel skips size and 7 bytes: 6, 7 and 3 for the three blobs

sd_write_aligned_blob (parser/parser_interface.c:274-284) pads so the table set starts at a multiple of 8 within the unit; the blob length includes the pad. unpack_dfa skips size & 7 bytes (linux:security/apparmor/policy_unpack.c:442-468).

The three DFAs travel as blobs: code six, a thirty-two bit length, then the bytes. But the writer wants the table set to start at a multiple of eight, counted from the start of this profile's buffer, so it puts zero bytes in front and counts them into the blob's length. Here: name aadfa, blob, length thirteen fifty, six zero bytes, and at offset fifty-six, the magic number of the table set. The pad isn't stored anywhere. The kernel recomputes it: a table set is always a multiple of eight long, so the pad is just the low three bits of the blob size. The blob length is little-endian; the size inside the table set is big-endian.

Details and sources

Details

  • sd_write_aligned_blob (parser_interface.c:274-284) writes the pad and the data as one blob; sd_serialize_dfa (parser_interface.c:383-396) writes the optional permission table and then the blob.
  • The kernel's formula in unpack_dfa is sz = blob - start - ((pos - start) & 7); pad = ALIGN(sz, 8) - sz, with pos already past the blob; that equals size & 7 (policy_unpack.c:442).
  • aa_dfa_unpack copies each table into its own allocation with unaligned reads (match.c:34), so the alignment of the data in the buffer does not matter to it.
  • In a buffer with several units the second unit does not start at a multiple of 8, so its DFAs are aligned to the unit, not to the buffer. In the 15416-byte example of the scene "One unit per profile" (a child and a hat; the same under both feature sets), the hat's unit starts at 0x1c96 and its policydb table set at 0x1dd6.
  • Byte order: the stream is little-endian (parser_interface.c:207-226), the table set big-endian (chfa.cc:399-415).
  • The pads and sizes on the slide were decoded from the 5009-byte stream of the example.
  • Kernel statements are read from the kernel source and were not run.

Read more

06 Permission rows and the exec name table

perms / permsv / xtable perms { version 1, array 8 [ row, row, ... ] } at 0b29, before the file DFA reserved0 allow0x244 deny0 subtree0 cond0 kill0 complain0 prompt0 audit0 quiet0 hide0 xindex0 tag0 label0 one row: 14 words of 32 bits, in this order permsv 3 after the file table: owner flag in accept2 a different profile, with three named exec targets (the profile is under Details): xtable: ARRAY count=3STRING len=13 "demo//worker\0"STRING len=11 ":ns2\0other\0"STRING len=12 "demo-helper\0" :ns2 NUL other: a target in another namespace with a permission table, written twice: inside policydb and after the file DFA; at most 12 names
The perms struct holds a version and an array of rows of 14 words each: reserved, allow, deny, subtree, cond, kill, complain, prompt, audit, quiet, hide, xindex, tag and label; permsv 3 follows the file table; a profile with named exec targets also has an xtable, an array of target names

perms: version 1 and an array of rows of 14 u32 (parser/parser_interface.c:346-374). permsv 3 follows the file table. xtable: names for exec slots 4 to 15, written inside policydb and after the file DFA (413-456, 590-602).

The DFAs don't travel alone. In front of the policydb blob and in front of the file blob sits a struct called perms: a version number, and an array of rows. Each row is fourteen thirty-two bit words. You know allow, deny, prompt, audit, quiet and xindex from the permissions deck. The others are zero. For our example the file table has eight rows, and after it comes permsv equals three, which tells the kernel that accept2 carries the owner flag. Give a profile exec rules with named targets and one more structure appears: xtable, an array of the target names. A row's xindex points into it.

Details and sources

Details

  • Row field order: reserved, allow, deny, subtree, cond, kill, complain, prompt, audit, quiet, hide, xindex, tag, label (parser_interface.c:346-362). The table is written when kernel_supports_permstable32_version >= 2 and it is not empty (parser_interface.c:387-390); permsv is written at parser_interface.c:376-380.
  • The row values on the slide are from the 5009-byte stream of the three-rule example.
  • In the kernel: unpack_perms_table (policy_unpack.c:925); unpack_perm refuses a version other than 1 (policy_unpack.c:905); unpack_pdb (policy_unpack.c:968): with a table the accept table is an index and accept2 is expected only when permsv is above 2 (policy_unpack.c:996); without a table both accept tables are required (policy_unpack.c:1002).
  • The xtable dump is from a different profile, compiled with apparmor_parser -Q -K -S and a feature set that has the kernel's policy entries (15416 bytes; the tree's abi/5.0 feature set gives the same size and the same table):
abi <abi/5.0>,
profile :ns1:demo /usr/bin/demo flags=(complain) {
  capability net_bind_service,
  set rlimit nofile <= 1024,
  /usr/bin/demo-helper Px -> demo-helper,
  /usr/bin/other Px -> :ns2:other,
  /usr/lib/demo/worker Cx -> worker,
  profile worker { /tmp/w r, }
  ^admin { /etc/demo/admin.conf rw, }
}
  • The three names are not in the order of the three rules: demo//worker (the Cx rule, written last) comes first and demo-helper (written first) comes last. The array is at 0x0bd8 inside policydb and again after the file DFA.
  • sd_serialize_xtable (parser_interface.c:413-456) writes nothing when slot 4 is empty (parser_interface.c:417-418); a target in another namespace is written as :ns, NUL, name, NUL (parser_interface.c:433-449). The index in a row is the slot minus 4 (policy_compat.cc:59-62). The table is written inside policydb and after the file DFA (parser_interface.c:590-602). The xmatch DFA has no table.
  • In the kernel: unpack_strs_table(e, "xtable", false, &pdb->trans) (policy_unpack.c:1075, policy_unpack.c:528); process_strs_entry refuses an entry that starts with NUL or white space and turns the NUL after a namespace back into : (policy_unpack.c:470); verify_perms checks each table index and trims unused tail entries (policy_unpack.c:1534).
  • Kernel statements are read from the kernel source and were not run.
  • Known discrepancy: array counts and string lengths are 16 bit with no check on the writer side (parser_interface.c:286-292, parser_interface.c:310-315); latent.
  • Known discrepancy: sd_serialize_xtable skips empty slots, which would shift later indexes if the table ever had a gap (parser_interface.c:417-430); not determined whether it can.
  • Known discrepancy, by title: "-> on a rule without exec or link permission is accepted and takes an exec table slot".

07 One unit per profile, child and hat

load_flattened_hats profile :ns1:demo ... { profile worker { ... } ^admin { ... } } a different profile (text under Details) 0000 1c96 3027 15416 bytes, three units 1 version, namespace "ns1", profile "demo" 2 version, profile "demo//admin" 3 version, profile "demo//worker" flag bit 1: a hat aa_unpack: loop over units until the buffer ends the 12 bytes 04 08 00 "version\0" 02 also mark where a unit starts known discrepancy: the second and third unit carry no namespace element
A profile in a namespace with a child profile and a hat is written as three units one after another, each with its own version element and the names demo, demo//admin and demo//worker; the kernel loops over the units until the buffer ends

__sd_serialize_profile writes a profile and then each child and hat as its own unit named parent//child (parser/parser_interface.c:683-791). aa_unpack loops over the units (linux:security/apparmor/policy_unpack.c:1745-1838).

In the profile text, a child profile or a hat sits inside its parent's braces. In the stream it doesn't. The writer serialises the parent, then calls the same function again for each child and hat, and each becomes a separate unit after the parent: its own version element, its own profile struct, and a flattened name, parent//child. A hat has one flag bit set to say so. Here's a profile in a namespace with a child and a hat: three units in a row. The kernel loops over the buffer until it runs out. The twelve bytes that open each unit, the name version plus the u32 code, also mark where a unit begins.

Details and sources

Details

abi <abi/5.0>,
profile :ns1:demo /usr/bin/demo flags=(complain) {
  capability net_bind_service,
  set rlimit nofile <= 1024,
  /usr/bin/demo-helper Px -> demo-helper,
  /usr/bin/other Px -> :ns2:other,
  /usr/lib/demo/worker Cx -> worker,
  profile worker { /tmp/w r, }
  ^admin { /etc/demo/admin.conf rw, }
}
  • Its output is 15416 bytes with units at 0x0000 (demo, with namespace ns1), 0x1c96 (demo//admin, flags 1, 0, 0) and 0x3027 (demo//worker). The first unit also has mode 1 (complain), capability allow 0x400 and rlimits with mask 0x80 and 16 values. The slide draws the bodies of the child and the hat as { ... }; with empty bodies the output is 13361 bytes and the units start at 0x0000, 0x1c96 and 0x2e5f.
  • In the kernel, aa_unpack (policy_unpack.c:1745) loops over the units; the version element is required only for the first unit (policy_unpack.c:1761). verify_header reports a namespace that differs from an earlier unit's as "invalid ns change" (policy_unpack.c:1442).
  • Who uses the 12 bytes \x04\x08\x00version\0\x02 as a marker. The kernel does not search for them: aa_unpack parses unit after unit in order (policy_unpack.c:1759). libapparmor splits a buffer at \x04\x08\x00version\0 (11 bytes) and writes the units one by one, for kernels without set_load (kernel_interface.c:68-124). The kernel uses its 12-byte aa_hdr_magic once, to recognise an uncompressed stream behind an 8-byte user header (apparmorfs.c:604, apparmorfs.c:649). The parser uses the same 12 bytes, followed by the version word, to accept a cache file that is not compressed (policy_cache.c:36-69).
  • Kernel statements are read from the kernel source and were not run.
  • Known discrepancies, by title: "Children and hats of a profile that names its namespace are serialised without the namespace"; "A child or hat that the kernel refuses is reported and then ignored: exit status 0"; "next_profile_buffer reads past the end of the policy buffer on kernels without set_load".

08 Compressed, and cached

zstd / cache 5009 bytes compress_policy_zstd 565 in all: 8-byte header + 557-byte frame (-S) 010300000000000028b52ffd header: version 1, level 3, padding zstd frame magic policy/compressed_load + policy/set_load both present: level 3 -O compress-default 10, -O compress-small 20, -c N explicit cache file 04 08 00 "version\0" 02 + version word not compressed: these 16 bytes are checked 01 LL 00 00 00 00 00 00 28 b5 2f fd compressed: header version, zstd magic and level LL are checked; one frame for all units known discrepancy: the help text and the man page give 10 as the default level
The 5009-byte stream compressed with zstd becomes 565 bytes behind an 8-byte header with version 1 and level 3; 565 bytes in all with the header; a cache file that is not compressed is accepted when its first 16 bytes are the version marker and the version word the parser would write now, a compressed cache file by its header version, zstd magic and level

With policy/compressed_load and policy/set_load the stream is zstd-compressed (level 3 unless -c, -O compress-default or -O compress-small): the 5009 bytes become 565 in all, an 8-byte header and a 557-byte frame. A cache file that is not compressed is accepted by its version marker and version word (parser/policy_cache.c:36-69); a compressed one by its header.

Two things can happen to the stream on its way. If the kernel's feature set says it can load compressed policy, the parser compresses each unit with zstd. Printed to standard output, our five thousand bytes become five hundred and sixty-five: an eight byte header with a version and the level, then the frame. The level you get without asking is three. The other thing is the cache. A cache file holds this stream as it is, or with compression as that header and one frame. Before using one, the parser checks how it starts: the version marker and the version word it would write today, or for a compressed file the header.

Details and sources

Details

  • set_compression_level (parser_main.c:480-506, constants at parser.h:98-100): an explicit -c level is kept; otherwise there is no compression unless the kernel has both policy/set_load and policy/compressed_load; then the level is 10 with -O compress-default, 20 with -O compress-small, else 3.
  • compress_policy_zstd compresses one unit (parser_interface.c:646-664). For -S and -o the output is struct compr_user_header (version 1, level, six bytes of padding) followed by the frame (lib.c:46-69, apparmor_private.h:41-47).
  • The 565 bytes and the first twelve bytes on the slide come from compiling the three-rule example with -S and a feature set that has the kernel's policy entries plus compressed_load: 565 bytes in all, an 8-byte header and a 557-byte frame. The header is written for -S, -o and cache files (parser_interface.c:757-763, policy_cache.c:336); a direct load writes the bare frame (parser_interface.c:747-756). The one-rule profile of 4529 bytes compresses to 497.
  • The cache. While a file is compiled, the temporary cache file receives the uncompressed units (parser_interface.c:768-770). Without compression that file is installed as it is. With compression, install_compressed_cache reads it back, compresses all units of the file into one zstd frame and installs the 8-byte header plus that frame (parser_main.c:1428-1435, policy_cache.c:311-343).
  • What the parser checks before using a cache file. A compressed file: is_valid_precompressed_profile looks at the header version, the zstd magic and the level (policy_cache.c:80-96, policy_cache.c:136-146, private.c:37-55); the version word is inside the frame and is not tested. A file that is not compressed: valid_cached_file_version compares the first 16 bytes, the 12-byte version marker and the version word the parser would write now (policy_cache.c:36-69, called at policy_cache.c:170-176). So a changed version word makes the uncompressed cache files stale; for compressed ones this check does not run.
  • In the kernel, policy_update looks for the zstd magic at offset 0 or after an 8-byte struct aa_user_hdr, or for the 12-byte version marker after such a header, and decompresses before unpacking (apparmorfs.c:598). The two feature entries exist only with CONFIG_SECURITY_APPARMOR_COMPRESSED_POLICY (apparmorfs.c:2642).
  • Errors: Compression error: out of memory, Compression error: %s (parser_interface.c:651, parser_interface.c:660). Warning, class WARN_DEBUG_CACHE: WARNING: Kernel does not support compressed policies. Defaulting to uncompressed (parser_main.c:501-503).
  • Kernel statements are read from the kernel source and were not run.
  • Known discrepancy: the default level is 3, while --help prints "default: 10" and apparmor_parser(8) says "a default value of 10 is used" (parser_main.c:490-498 against parser_main.c:267-277; apparmor_parser.pod:376).
  • Known discrepancies, by title: "Cache hit on an uncompressed cache loads the profile twice and writes the cache without -W"; "-n does not skip the policy cache, so a cached profile can be loaded into the wrong namespace".
  • Not determined: the kernel's limits on decompressed size. Cache directory layout and time stamp rules were not re-verified for this deck.

09 The kernel reads it back

aa_unpack write to .load / .replace policy_updatedecompress aa_unpackper unit verify_headerkernel ABI 5 to 9 unpack_profilefields by name unpack_pdbperms, blob, names unpack_dfaskip the pad aa_dfa_unpacktables copied out checked at load every access inside the buffer (aa_inbounds) table sizes agree accept index, and index + 1, inside the perms table no row with a bit in both allow and deny; xindex in range verify_dfa when aa_g_paranoid_load is set base + 255 < table size next, check, default < states diff chains end flag bits known taken as given the table set's version string and total size whether the tables match any profile text: there is none known discrepancy: accept indexes of an xmatch DFA that brings its own permission table are not range-checked; this parser writes no such table
The kernel load path as a chain of functions from the write to .load or .replace through policy_update, aa_unpack, verify_header, unpack_profile, unpack_pdb, unpack_dfa and aa_dfa_unpack; a list of what is checked at load, the checks of verify_dfa under paranoid load, and what is taken as given

Load path: policy_update (decompress), aa_unpack (units), verify_header, unpack_profile, unpack_pdb, unpack_dfa, aa_dfa_unpack, then verify_dfa, verify_dfa_accept_index and verify_perms.

The write lands in policy_update, which decompresses. aa_unpack walks the buffer unit by unit. verify_header checks that the kernel ABI in the version word is between five and nine. unpack_profile asks for each field by name, and aa_dfa_unpack copies each table out into its own allocation. Then comes verification. Every access to the buffer was bounds-checked on the way. With paranoid load on, verify_dfa checks that every base plus two fifty-five is in range, that state numbers are in range, and that diff chains end. Accept values must be valid rows. No row may both allow and deny a bit. What the kernel can't check is whether the tables mean what your profile said. So who may write policy matters.

Details and sources

Details

  • The chain: policy_update detects compression and decompresses (apparmorfs.c:598); aa_unpack loops over the units (policy_unpack.c:1745); verify_header checks the version word and the namespace (policy_unpack.c:1442); unpack_profile takes the fields in order (policy_unpack.c:1121); unpack_pdb takes the permission table, the DFA blob and the name table (policy_unpack.c:968); unpack_dfa skips the pad (policy_unpack.c:442); aa_dfa_unpack checks the magic and copies the tables out (match.c:333).
  • verify_dfa (match.c:154) runs when aa_g_paranoid_load is set (policy_unpack.c:458, lsm.c:2006).
  • verify_dfa_accept_index checks, for the file and policydb DFAs, that the accept value, and the accept value plus 1 with the owner flag, are inside the table; verify_perms checks all three tables (policy_unpack.c:1492); verify_perm rejects a row with a bit in both allow and deny (policy_unpack.c:1511).
  • aa_dfa_unpack does not use the version string or th_ssize of the table set header.
  • A kernel ABI below 5 or above 9 gives "unsupported interface version".
  • Kernel statements are read from the kernel source and were not run. Nothing in this deck was loaded into a kernel.
  • Known discrepancy: verify_profile calls verify_dfa_accept_index for rules->file and rules->policy only. For attach.xmatch with its own perms struct, aa_lookup_perms would index without a bound (policy.h:156, domain.c:349). Reaching it needs a hand-built stream and the right to administer policy; not reproduced.

10 Where to go from here

ROADS text tree states permissions fewer states tables bytes Unpacking and verifying policythe kernel side Matching at run timeusing the tables The policy cachewhere these bytes are kept Loading policy into the kernelhow they get written Who can change policywho may write them at all
The journey from text through tree, states, permissions, fewer states and tables to bytes, and the roads onward: unpacking and verifying policy, matching at run time, the policy cache, loading policy into the kernel and who can change policy

Next: unpacking and verifying policy, matching at run time. Operations: the policy cache, loading policy into the kernel. Security: who can change policy. Back: table compression, the backend pipeline.

That's the end of the road that started with three lines of profile text: a tree, thirty-eight states, their permissions, twenty-two states, tables, and now five thousand and nine bytes, or five hundred and sixty-five compressed. From here the story continues in the kernel: later decks go through the unpacking in detail, and through matching, which uses the tables. If you operate systems, later decks on the cache and on loading cover where these bytes are kept and how they get written. And who gets to write them at all is a question for a later deck on policy administration.

Details and sources

Details

Read more

Basis of the statements on this page: "tree" means the parser built from the AppArmor commit 836052d21 (6.0.0~alpha2) was run with -Q -K -S, and the output was decoded with a short script written for the purpose (not code from the tree); nothing was loaded into a kernel. The three-rule example was compiled with a feature set that has the policy entries the kernel exports (version word 0x202007); the other examples with the tree's abi/5.0 file (version word 5). "code" means the parser source, or the kernel source, was read and not run; "derived" means the result was reasoned from that code. Nothing here was observed on a running kernel.

Element encoding

CodeNameForm
0, 1, 2, 3U8, U16, U32, U64the code byte, then the integer, little-endian
4NAME04, 16-bit length including the NUL, bytes; precedes the value it names
5STRING05, 16-bit length including the NUL, bytes
6BLOB06, 32-bit length, bytes
7, 8STRUCT, STRUCTEND07, elements, 08
11, 12ARRAY, ARRAYEND0b, 16-bit count, elements, 0c
9, 10, 13LIST, LISTEND, OFFSETdefined, not written

Code: parser_interface.c:162-331; kernel readers at policy_unpack.c:242 and policy_unpack.c:291.

The version word

BitsMeaning
0-9kernel ABI: 7 or 6 with policy/versions/v7 or v6, else 5
11force complain (only when the kernel ABI is above 5)
12-19parser ABI, 2 (only above 5)
20-27policy version, 2 (only above 5)
  • 0x202007 with the kernel's policy entries; 5 with the tree's abi/5.0 (tree).
  • The kernel refuses a kernel ABI below 5 or above 9: "unsupported interface version" (code).

Profile struct

#FieldWritten when
1string: name (parent//child when flattened)always
2blob aadfa (xmatch), u32 xmatch_lenxmatch exists
3string disconnectedattach_disconnected.path=
4string disconnected_ipcset, and feature domain/disconnected.ipc
5u32 killset, and feature domain/kill.signal
6u32 errorset, and feature policy/profile/error
7struct flags: flag bits, mode, auditalways
8u32 path_flagsany path flag
9four u32: capability allow, audit, quiet, 0always
10struct caps64always
11array identityfeature domain/identity
12struct xattrsxattrs=
13struct rlimits: mask, 16 u64any rlimit
14array net_allowed_afold network encoding only
15struct policydb: perms, blob aadfa, xtablepolicydb DFA exists
16perms, permsv, blob aadfa (file DFA)file DFA exists
17struct xtablea named exec target
18struct endalways
  • Writer: parser_interface.c:480-604; reader: policy_unpack.c:1121 (code). The line numbers of each field are in the details of the scene "The profile struct, field by field".
  • Read by the kernel and not written by this parser: rename, attach, capsx, secmark, tags, start, dfa_start, a data struct (code).
  • Written only for a feature set with the entry, and not read by the kernel at this commit: fields 4, 6, 11, 14 (code).

The running example decoded

OffsetElementValue
0000name version, u3204 08 00 "version\0" 02 07 20 20 00 = 0x202007
0010name profile, struct
001cstringdemo
0024name aadfa, blobsize 1350 = 6 pad + table set of 1344 at 0038 (xmatch: 15 states, accept[14] = 1)
0578u32xmatch_len 13
057dname flags, struct of 3 u320, 0 (enforce), 0
05974 u32capability allow, audit, quiet, 0: all 0
05abname caps64, struct of 4 u32all 0
05cbname policydb, struct
05d8name perms, struct: version 1, array of 2 rowsallow 0x8000, 0x8004
0683name aadfa, blobsize 1175 = 7 pad + table set of 1168 at 0698 (5 states)
0b28struct end (policydb)
0b29name perms, struct: version 1, array of 8 rowsallow 0x200 four times, 0x244 twice, 0x10201 twice
0d78name permsv, u323
0d87name aadfa, blobsize 1531 = 3 pad + table set of 1528 at 0d98 (22 states)
1390struct end (profile)total 0x1391 = 5009

Tree. Table sets: 1344 + 1168 + 1528 = 4040 bytes; permission tables 140 + 560 bytes; everything else 269 bytes.

DFA blobs

  • The blob is the pad followed by the table set; the pad makes the table set start at a multiple of 8 from the start of the unit (parser_interface.c:274-284) (code).
  • The kernel skips size & 7 bytes (policy_unpack.c:442) (code).
  • The stream is little-endian, the table set big-endian (code).

Permission table

  • perms { version 1, array N [ row ... ] }; a row is 14 u32: reserved, allow, deny, subtree, cond, kill, complain, prompt, audit, quiet, hide, xindex, tag, label (parser_interface.c:346-374) (code).
  • permsv 3 follows the file table; above 2 the kernel expects an accept2 table with the owner flag (code).
  • Kernel checks: version 1; each accept index inside the table; no bit in both allow and deny (code).

Exec name table

  • xtable: an array of strings for exec slots 4 to 15; a row's xindex is the slot minus 4 (code).
  • A target in another namespace is written :ns, NUL, name, NUL; the kernel turns the NUL back into : (code).
  • Written inside policydb and after the file DFA when permission tables are used (tree).

Units

  • A profile, and after it each child and hat, is one unit: version, an optional namespace, and the profile struct (code).
  • A child or hat is named parent//child; a hat has flag bit 1 (code, tree).
  • The 12 bytes 04 08 00 "version\0" 02 mark the start of a unit (code).

Compression

Optionzstd level
none, kernel has policy/set_load and policy/compressed_load3
-O compress-default10
-O compress-small20
-c NN
kernel lacks either entry, no -cnot compressed
  • Header in front of the frame: version 1, level, six bytes of padding (code). The example: 5009 bytes become 565 with -S (the 8-byte header and a 557-byte frame; a direct load writes the bare frame), starting 01 03 00 00 00 00 00 00 28 b5 2f fd (tree).
  • The kernel finds the zstd magic at offset 0 or after the 8-byte header (code).

Cache file check

  • Without compression a cache file is the uncompressed stream. The parser accepts it only if its first 16 bytes are the 12-byte version marker and the version word the parser would write now (policy_cache.c:36-69) (code).
  • With compression (policy/compressed_load and policy/set_load) a cache file is the 8-byte header and one zstd frame over all units of the file. The parser accepts it by the header version, the zstd magic and the level; the version word is not tested (policy_cache.c:80-96, policy_cache.c:148-181) (code).

What the kernel verifies

CheckFunctionWhen
every access inside the bufferaa_inboundsalways
kernel ABI field of the version word 5 to 9; no namespace change between unitsverify_headeralways
magic number and header flags of a table setaa_dfa_unpackalways
table sizes agreeaa_dfa_unpackalways
base + 255 inside next/check; state numbers in range; diff chains endverify_dfaparanoid load
accept index, and index + 1 with the owner flag, inside the tableverify_dfa_accept_indexfile and policydb DFAs
xindex in range; no bit in both allow and denyverify_perms, verify_permalways

All rows: code.

Kernel error strings

"unsupported interface version", "invalid ns change", "bad xmatch", "missing xmatch len", "profile missing flags", "failed to unpack profile flags", "failed to unpack profile capabilities", "failed to unpack policydb", "failed to unpack end of profile". They appear in the info field of the audit record (code).

Known discrepancies

  • With the tree's abi/5.0 file as kernel features the parser writes version word 5: the lookup of policy/versions/v7 matches the end of notify_versions (features.c:361-444) (tree).
  • The default compression level is 3; the help text and the man page say 10 (code).
  • Accept indexes of an xmatch DFA with its own permission table are not range-checked by the kernel (code; not reproduced; this parser writes no such table).
  • Array counts and string lengths are 16 bit with no check on the writer side (code; latent).
  • sd_serialize_xtable skips empty slots (code; not determined whether a gap can occur).
  • By title: "attach_disconnected.ipc is dropped without a warning and its bit is written anyway"; "mediate_deleted is packed with the bit the kernel calls PATH_DELEGATE_DELETED"; "-> on a rule without exec or link permission is accepted and takes an exec table slot"; "Children and hats of a profile that names its namespace are serialised without the namespace"; "A child or hat that the kernel refuses is reported and then ignored: exit status 0"; "next_profile_buffer reads past the end of the policy buffer on kernels without set_load"; "Cache hit on an uncompressed cache loads the profile twice and writes the cache without -W"; "-n does not skip the policy cache, so a cached profile can be loaded into the wrong namespace".

Not determined

  • Whether kernels that advertise policy/notify_versions treat a version 5 stream differently.
  • Cache directory layout, time stamp rules and the recompress path.
  • The kernel's limits on decompressed size.
  • Whether the exec table can have an empty slot between two used ones.

Roads from here

START HERE

PROFILE STRUCTURE

  • Hats — hats in the profile language
  • Child profiles — child profiles in the profile language

SECURITY · threat model and limits

RUNNING THE PARSER

LIBAPPARMOR

IN THE KERNEL

THE DFA ENGINE · libapparmor_re