include if exists

optional includes, .d dirs for application developers for administrators for security researchers

include if exists is an include that is not an error when the file is missing. When the name is a directory every regular file in it is included. Shipped policy ends its tunables files and abstractions with such a line for a NAME.d directory and that is where site changes go.

On the map: preamble and reuse

▶ Watch this deck — 8 scenes, ~6 min

Worth seeing first

  • include — the plain include line this one extends

01 A file that may not be there

INCLUDE IF EXISTS profile demo /usr/bin/demo { include <abstractions/base> include <local/demo> } /etc/apparmor.d/local/demo not installed Could not open 'local/demo' exit status 1: the packaged profile no longer loads include if exists <local/demo> exit status 0, no message $ apparmor_parser -p demo | grep local ##failed include <local/demo> /etc/apparmor.d/local/demo /srv/demo-extra/** r, when the file appears, its rules are in
Demo's profile includes the file local/demo, which is not installed. With a plain include the parser reports Could not open 'local/demo' and exits with status 1, so the packaged profile no longer loads. With include if exists the same profile compiles with exit status 0, and apparmor_parser -p shows the marker ##failed include for local/demo. When the file /etc/apparmor.d/local/demo appears, its rules are part of the profile.

include if exists <local/demo>: taken when the file is there, skipped without an error when it is not.

Demo's profile comes from a package, and the administrator of one machine wants to add a rule without editing the packaged file. So the profile includes a local file. But on most machines that file doesn't exist, and a plain include of a missing file is an error: the whole profile fails to load. Three words fix it: include if exists. Now a missing file is skipped quietly. The option -p shows the skipped include as failed include, which here just means not there. And the day somebody creates the file, its rules become part of the profile.

Details and sources

Details

  • Run with the tree's parser for this deck: a profile whose line 3 is include <local/demo> gives AppArmor parser error for /work/inc-missing.sd in profile /work/inc-missing.sd at line 3: Could not open 'local/demo', exit status 1. /work/inc-missing.sd is the scratch name of the file.
  • With include if exists <local/demo> the example file of this district compiles with exit status 0 and the same 106 compiled lines, and -p prints ##failed include <local/demo> (run with the tree's parser for this deck).
  • The last step is derived: no run had a local/demo file, because the tree used for the runs cannot be changed. /srv/demo-extra/** r, is an invented line. A run with a file that is present is in the scene "Your change goes in your file".
  • 358 of the 1126 include lines under profiles/apparmor.d of the tree are include if exists.
  • The tree ships profiles/apparmor.d/local/ with a README only.

02 Three words, then the file name

GRAMMAR include #include if exists <magic/path> "path" path no quotes: opened as written no comma include if exists <local/demo> line 8 of demo's file one unit: blanks between the words, and a blank after exists include if exists <tunables/home.d> the name may be a directory #include if exists "../does-not-exist/does-not-exist" quoted, missing, no error (a parser test) include if exists <x> if ${DEMO_DEBUG} { ... } not the if of conditional blocks: nothing is evaluated here
Syntax diagram: the three words include, if, exists, or #include if exists, then a magic path in angle brackets, a quoted path or a bare path, and no comma. The three words are one unit with blanks between them and a blank after exists. Examples: demo's line include if exists local/demo, a directory name tunables/home.d, and a quoted missing path from the parser's tests. It is not the if of conditional blocks: nothing is evaluated.

include if exists <magic/path> or "path". Same name forms and search as include; no comma.

The grammar is the include line with two extra words: include, if, exists, then the file name in angle brackets or quotes. The three words belong together, with blanks between them and a blank after the last. Everything you know about include still holds: brackets search, quotes don't, no comma, and the hash spelling works. The name may also be a directory, which we'll get to. A missing file is no error, in either spelling. And don't confuse this if with the if of conditional blocks. There's no expression here, only the question whether the file can be opened.

Details and sources

Details

  • The three words are one unit: blanks between them and a blank after exists.
  • The file name forms, the search for a magic path and the missing comma are those of include: see include.
  • if and exists are not keywords anywhere else in this position.
  • The preamble takes the line too: #include if exists <includes-preamble-does-not-exist> (include_tests/preamble_good_6.sd, a test the parser is expected to accept).
  • The quoted, missing path on the slide is from include_tests/ok_85.sd, also expected to be accepted.
  • A file already included in this profile is skipped, as for include.
  • #include if exists gives the deprecated use of '#include' warning with --warn=pound-include.

for AppArmor developers for security researchers

In the code

03 Name a directory and every file in it is included

A DIRECTORY include if exists <tunables/home.d> the last line of tunables/home /etc/apparmor.d/tunables/home.d/ site.local included nfs included $ apparmor_parser -p demo | grep home.d ##included <tunables/home.d> README .hidden site.local.dpkg-old skipped, silently site.local~ nfs.orig skipped, with a line: Ignoring: '.../nfs.orig' archive/ a subdirectory: not entered do not make two files of one directory depend on each other
The last line of tunables/home, include if exists tunables/home.d, names a directory. Of the entries in /etc/apparmor.d/tunables/home.d the files site.local and nfs are included; README, .hidden and site.local.dpkg-old are skipped silently; site.local~ and nfs.orig are skipped with a line Ignoring; the subdirectory archive is not entered. apparmor_parser -p shows the marker ##included tunables/home.d.

A directory include takes every regular file in it. Hidden files, README, package leftovers and backups are skipped.

The name after include doesn't have to be a file. Give it a directory and the parser includes every regular file in it. That's the whole mechanism behind the .d directories you see next to shipped files. Not quite every file, though. Hidden files, a file called README and the leftovers that package managers create are skipped silently. Editor backups that end in a tilde, and .orig and .rej files, are skipped with a line that says ignoring. Subdirectories aren't entered. One piece of advice: don't make two files of one directory depend on each other.

Details and sources

Details

  • Skipped silently: names that start with ., the name README, and names ending in .dpkg-new, .dpkg-old, .dpkg-dist, .dpkg-bak, .dpkg-remove, .pacsave, .pacnew.
  • Skipped with Ignoring: '<path>' on standard error: names ending in .rpmnew, .rpmsave, .orig, .rej and ~.
  • A symbolic link is followed and must lead to a regular file. Subdirectories are not entered.
  • A file already included in this profile is skipped.
  • The line works without if exists too; a missing directory is then an error.
  • The directory listing on the slide is an illustration derived from these skip rules; it was not run. The tree ships tunables/home.d/site.local only.
  • The -p line is from a run with the tree's parser for this deck, on the example file of this district.
  • A parser test: a directory holding bad-include.dpkg-new, bad-include~ and one good-include compiles (include_tests/ignored_suffix_1.sd).
  • Known discrepancy: The files of an included directory are compiled in reverse alphabetical order, and an error in one is reported with a neighbour's file name.

for AppArmor developers for security researchers

In the code

04 Your change goes in your file

NAME.d tunables/home @{HOMEDIRS}=/home/ @{HOME}=@{HOMEDIRS}/*/ /root/ include if exists <tunables/home.d> tunables/home.d/site.local #@{HOMEDIRS}+=/srv/nfs/home/ /mnt/home/ as shipped: a comment profile demo audit deny @{HOME}/.ssh/** rw, @{HOMEDIRS} += /srv/nfs/home/ $ apparmor_parser -Q -D rule-exprs demo before {/home/*,/root}/.ssh/** after {/home/*,/root,/srv/nfs/home/*}/.ssh/** every profile that includes tunables/global now knows the third place; the shipped tunables/home is untouched
The shipped file tunables/home defines HOMEDIRS and HOME and ends with include if exists tunables/home.d. The file tunables/home.d/site.local, shipped as a comment, gains the line @{HOMEDIRS} += /srv/nfs/home/. Demo's rule audit deny @{HOME}/.ssh/** rw is unchanged. The rule dump shows it compiled as {/home/*,/root}/.ssh/** before and {/home/*,/root,/srv/nfs/home/*}/.ssh/** after.

Add a file to tunables/home.d/ with @{HOMEDIRS} += /srv/nfs/home/. Every rule written with @{HOME} follows.

This machine keeps home directories on a file server, under /srv slash nfs /home. Demo's rule about .ssh is written with the @{HOME} variable, which comes from the shipped file tunables/home. You could edit that file, but then you'd carry a change inside a shipped file. Instead, look at its last line: include if exists, tunables/home.d. Put one line into a file in that directory, adding your path to the @{HOMEDIRS} variable with +=. Compile, and the rule covers three places instead of two, in every profile that includes the tunables.

Details and sources

Details

  • Run with the tree's parser for this deck. Before: the example file of this district, whose -D rule-exprs dump has aare: {/home/*,/root}/.ssh/**. After: aare: {/home/*,/root,/srv/nfs/home/*}/.ssh/**, exit status 0.
  • How the second run was made: the one-line file site.local (@{HOMEDIRS} += /srv/nfs/home/) stood in a directory given with -I and was included through include if exists <site.local> in the profile's own preamble, after include <tunables/global>. It stands in for a file in tunables/home.d/, which could not be added to the tree used for the runs. -p of that run shows ##included <site.local> followed by @{HOMEDIRS} += /srv/nfs/home/.
  • The commented line in the middle box is the shipped text of home.d/site.local. The left box is tunables/home.
  • += is needed because = on a variable that is already declared is an error: see Variable assignment.
  • Shipped .d directories with a site.local of comments: tunables/home.d, tunables/multiarch.d, tunables/xdg-user-dirs.d.
  • Every tunables file that tunables/global includes ends with include if exists <tunables/NAME.d>, and shipped abstractions end with include if exists <abstractions/NAME.d> (base:184).
  • The policy cache is rebuilt after such a change, because included files count for its time stamp: see The policy cache (derived, not run).
  • site.local is itself a file of the tree; whether a package upgrade leaves a changed copy alone depends on the distribution's packaging, which was not examined.

05 A directory that every profile reads for security researchers

EXTENSION POINT abstractions/base include if exists <abstractions/base.d> demo demo-helper ... ... ... ... abstractions/base.d/ anything whatever lands there is in every profile that includes base tunables/global.d/ @{HOME} += / in the tunables, one line widens every rule that uses the variable @{HOMEDIRS} += /srv/nfs/home/, in profile .../site-bad.local at line 1: Variable declarations do not accept trailing commas no profile that includes it compiles a missing directory prints nothing a file under the wrong name prints nothing: -p is the only place it shows
abstractions/base ends with include if exists abstractions/base.d, and six profiles include base. A file dropped into abstractions/base.d is part of every one of them. In the same way a line such as @{HOME} += / in tunables/global.d widens every rule that uses the variable. A site file with a trailing comma stops compilation: Variable declarations do not accept trailing commas. A missing directory prints nothing, and a file under the wrong name prints nothing.

Files in abstractions/NAME.d/ and tunables/NAME.d/ are compiled into every profile that includes NAME. Absence is silent.

Look at this from the other side. The last line of the base abstraction includes a directory if it exists. Whatever lands in that directory becomes part of every profile that includes base, and many profiles do. The same holds for the tunables: one line there can widen the @{HOME} variable for the whole system. Write access to these directories is write access to policy. A broken file there stops every profile that pulls it in from compiling. And the mechanism is silent by design: an absent file and a misnamed file both produce no message at all.

Details and sources

for security researchers

Details

  • The last line of the base abstraction is include if exists <abstractions/base.d> (base:184); tunables/global ends with include if exists <tunables/global.d>.
  • The chips anything and @{HOME} += / are illustrations derived from the include mechanism. No run added a file to the tree's .d directories.
  • 102 of the 198 profile files directly under profiles/apparmor.d of the tree include abstractions/base (counted for this deck with grep).
  • The error, run with the tree's parser for this deck: AppArmor parser error for /work/ie-bad.sd in profile /work/site-bad.local at line 1: Variable declarations do not accept trailing commas, exit status 1. The file site-bad.local held @{HOMEDIRS} += /srv/nfs/home/, and was included through include if exists from a directory given with -I.
  • ##failed include <name> in -p output is the only trace of a missing magic path.
  • The directories are under /etc/apparmor.d, written by root and by packages: see Who can change policy.
  • A directory include follows symbolic links to regular files. Which names are skipped: see the scene "Name a directory and every file in it is included".
  • Known discrepancy: -p prints ##included "PATH" for a quoted include that does not exist.
  • Known discrepancy: An include inside an if or qualifier block makes the same include in the rest of the profile a no-op.

for security researchers

Attack surface

  • An optional include is a standing invitation: a profile with include if exists <local/NAME> compiles with and without the file, and neither its presence nor its absence is reported.
  • A review of a profile must include the .d directories of everything it includes, and the -I directories of the parser's invocation.
  • How the utilities treat includes is in aa-logprof.

for security researchers

Read more

06 One more scanner state and a flag for AppArmor developers for security researchers

IN THE SCANNER include/{WS} INCLUDE if_exists = false include{WS}+if{WS}+exists/{WS} INCLUDE_EXISTS if_exists = true include_filename(name, search, if_exists) opened, a regular file push a buffer opened, a directory dirat_for_each() not opened yyerror("Could not open '%s'") or, with if_exists: return include_dir_cb() for each entry of the directory: skip if in g_includecache skip unless a regular file yypush_buffer_state() per file the longer pattern must match completely: "include if exists" without a blank after it falls back to an include of a file called "if"
Two scanner patterns. include/{WS} pushes the state INCLUDE with if_exists false. include{WS}+if{WS}+exists/{WS} pushes the state INCLUDE_EXISTS with if_exists true. Both call include_filename(name, search, if_exists). A regular file gets a new buffer; a directory is walked with dirat_for_each; when nothing is opened, yyerror reports Could not open, or with if_exists the function returns. include_dir_cb skips files in g_includecache and entries that are not regular files and pushes a buffer per file.

parser_lex.l:789-794 is the pattern, 192-194 the quiet return, 128-160 the directory callback.

In the scanner this is a second pattern next to the one for include: the three words with blanks between them, pushing a state of their own. Both states end in the same function, include_filename, and the only thing that differs is one flag. When nothing can be opened, the plain include raises an error and this one just returns. If what was opened turns out to be a directory, a callback runs for each entry and pushes a buffer for each regular file that isn't in the include cache. And since it's a longer pattern, it only wins when it matches completely.

Details and sources

for AppArmor developers for security researchers

Details

  • The pattern and its state: parser_lex.l:789-794. include_filename(): parser_lex.l:162-225; the quiet return: parser_lex.l:192-194. The directory callback include_dir_cb(): parser_lex.l:128-160.
  • The directory is listed with alphasort (_aa_dirat_for_each, private.c:516). Every file of the directory is pushed before any is parsed.
  • Known discrepancy: The files of an included directory are compiled in reverse alphabetical order, and an error in one is reported with a neighbour's file name. Recorded in the research notes: a directory with 10-define (@{V} = a) and 20-extend (@{V} += b) gives Failed to find declaration for: V and an error naming /work/10-define at line 1.
  • Messages for directories: Could not open '%s' in '%s' and Could not process include directory '%s' in '%s' (parser_lex.l:151, parser_lex.l:214-215).

for AppArmor developers for security researchers

Read more

07 Mistakes, most of them silent

MISTAKES include if exist <local/demo> Could not open 'if' include if exists<local/demo> Could not open 'if' a blank is needed after exists include if exists <local/dmeo> no message, exit status 0, no rules check with -p: ##failed include <local/dmeo> @{HOMEDIRS} += /srv/nfs/home/, a site file in profile .../site-bad.local at line 1: Variable declarations do not accept trailing commas if defined @{EXTRA} { include <demo-extra> } Could not open 'demo-extra' a conditional block does not make an include optional if exists forgives a missing file, not a broken one
Five lines and their results. include if exist, without the s: Could not open 'if'. include if exists with no blank before the bracket: Could not open 'if'. A misspelt file name local/dmeo: no message, exit status 0, no rules; -p shows ##failed include. A site file with a trailing comma: Variable declarations do not accept trailing commas. A plain include inside a conditional block: Could not open 'demo-extra'.

Could not open 'if' means the three words were not recognised. A misspelt file name is silent: check with -p.

The loud kind of mistake: write exist without the s, or leave out the blank before the bracket, and the parser no longer sees the three-word form. It sees a plain include of a file called if, and tells you it can't open if. The quiet kind is worse. Misspell the file name and nothing happens, no message and no rules, because a missing file is exactly what you allowed. The option -p will show the failed include. And only absence is forgiven: a site file with a syntax error still stops the compile, and so does a plain include of a missing file inside a conditional block.

Details and sources

Details

  • All five rows were run with the tree's parser; rows 1, 2 and 4 for the plans of this district, rows 3 and 5 for this deck with a small profile for demo. Rows 1, 2, 4 and 5 exit with status 1.
  • Rows 1 and 2 in full: AppArmor parser error for FILE in profile FILE at line N: Could not open 'if'. include if exists at the end of a line, with no file name, gives the same message (recorded in the research notes).
  • Row 3: exit status 0 and no message, also with --warn=all apart from warnings that do not concern the include; -p prints ##failed include <local/dmeo>.
  • Row 4 in full: AppArmor parser error for /work/ie-bad.sd in profile /work/site-bad.local at line 1: Variable declarations do not accept trailing commas. The file was included through include if exists from a directory given with -I.
  • Row 5 in full: AppArmor parser error for /work/ie06e.sd in profile /work/ie06e.sd at line 3: Could not open 'demo-extra'. @{EXTRA} is not defined in that file. The research notes record the same for if false { include <does/not/exist> }.
  • Includes are resolved before any block is evaluated, so an include in a block that is not taken must still exist: see Conditional blocks.
  • The file names under /work/ are scratch names of the runs.

08 Where the optional files live

ROADS include if exists <tunables/home.d> syn-include the plain form syn-abstractions-tunables which files end with this line syn-variable-assignment += in a site file syn-variables what a site file extends sec-policy-admin who can write there adm-parser -p syntax-map
The line include if exists tunables/home.d in the centre, with roads to the decks syn-include and syn-abstractions-tunables on the left, syn-variable-assignment and syn-variables on the right, and sec-policy-admin, adm-parser and syntax-map below.

Onward: abstractions and tunables, variable assignment, who can change policy.

That's include if exists: an include that forgives absence, and the reason .d directories work. The shipped files that end with such a line are in the deck on abstractions and tunables. The operator you'll use in a site file, +=, is in the deck on variable assignment. And who's allowed to write to these directories is a question for the deck on who can change policy.

Details and sources

Read more

Basis of the statements on this page: "tree" means the parser built from the AppArmor commit 836052d21 (6.0.0~alpha2) was run on the fragment (-Q -K --skip-kernel-load, with the tree's abi/5.0 file as kernel feature set and the tree's profiles/apparmor.d on the search path), with nothing loaded into a kernel; "code" means the parser or library source was read and not run; "derived" means the result was reasoned from that code. Nothing here was observed on a running kernel.

Grammar

INCLUDE IF EXISTS:  ( 'include' | '#include' ) 'if' 'exists' ( <magic/path> | "path" | path )
  • The three words are separated by blanks, and a blank follows exists (tree: without it the parser reports Could not open 'if').
  • File name forms, search path and the missing comma are those of include.
  • The one difference from include: a name that cannot be opened is not an error (tree).
  • The line may stand in the preamble and in a profile body (tree; the tests include_tests/preamble_good_6.sd and ok_85.sd).

Outcome

The name isincludeinclude if exists-p marker
a fileits text is includedthe same##included <name>
a directoryevery regular file in it is includedthe same##included <name>
missingCould not open 'NAME', exit status 1 (tree)nothing, exit status 0 (tree)##failed include <name>
a file with an error in itthe error, reported for the included file, exit status 1the same (tree)none: the compile stops

Directory includes

  • Every regular file in the directory is included; a symbolic link is followed and must lead to a regular file; subdirectories are not entered (code).
  • Skipped silently: names starting with ., README, and names ending in .dpkg-new, .dpkg-old, .dpkg-dist, .dpkg-bak, .dpkg-remove, .pacsave, .pacnew (code).
  • Skipped with Ignoring: '<path>' on standard error: names ending in .rpmnew, .rpmsave, .orig, .rej, ~ (code).
  • A file already included in the same preamble or profile is skipped (code).
  • Do not make two files of one directory depend on each other.

The NAME.d convention

  • Each tunables file that tunables/global includes, and tunables/global itself, ends with include if exists <tunables/NAME.d>. Shipped abstractions end with include if exists <abstractions/NAME.d> (the last line of abstractions/base is line 184).
  • Of the .d directories the running example tries, three exist in the tree: tunables/home.d, tunables/multiarch.d, tunables/xdg-user-dirs.d, each with a site.local of comments. The others give ##failed include (tree).
  • A worked site change (tree): the one line @{HOMEDIRS} += /srv/nfs/home/ changes demo's compiled rule from {/home/*,/root}/.ssh/** to {/home/*,/root,/srv/nfs/home/*}/.ssh/**. In the run the line stood in a file site.local in a directory given with -I and was included with include if exists <site.local> after include <tunables/global>, standing in for tunables/home.d/site.local.

Errors

MessageCauseBasis
Could not open 'if'exist for exists, or no blank after exists, or no file nametree
Could not open 'NAME'a plain include of a missing file, also inside a conditional block that is not takentree
in profile .../site-bad.local at line 1: Variable declarations do not accept trailing commasa syntax error in an optional file that existstree
no messagea misspelt name after include if existstree
Ignoring: '<path>'a backup or reject file in an included directory; not an errorcode

for security researchers

Attack surface

  • abstractions/NAME.d/ and tunables/NAME.d/ are extension points for every profile that includes NAME. Write access to them is write access to policy.
  • Presence and absence are both silent. ##failed include in -p output is the only trace of a missing magic path.
  • A broken file in such a directory stops every profile that includes NAME from compiling.
  • A review of a profile includes the .d directories of everything it includes and the -I directories of the parser's invocation.
  • These points are reasoned from the mechanism; no run added a file to the tree's .d directories.

for AppArmor developers for security researchers

In the code

  • Patterns and states: include{WS}+if{WS}+exists/{WS} pushes INCLUDE_EXISTS (parser_lex.l:789-794); the #include form (parser_lex.l:781-787).
  • include_filename(name, search, if_exists) (parser_lex.l:162-225); file or directory by fstat() (parser_lex.l:199-217).
  • include_dir_cb() (parser_lex.l:128-160) and libapparmor's directory walk (private.c:498-565); the skip lists (private.c:100-121, private.c:162-183, parser_misc.c:71-79).
  • Known discrepancy: An include inside an if or qualifier block makes the same include in the rest of the profile a no-op. Recorded in the research notes.
  • Known discrepancy: The files of an included directory are compiled in reverse alphabetical order, and an error in one is reported with a neighbour's file name. Recorded: 10-define with @{V} = a and 20-extend with @{V} += b give Failed to find declaration for: V and an error naming /work/10-define at line 1.
  • Known discrepancy: -p prints ##included "PATH" for a quoted include that does not exist. Recorded in the research notes.

Roads from here

START HERE

PREAMBLE AND REUSE

SECURITY · threat model and limits

RUNNING THE PARSER

INSIDE THE PARSER

  • The lexer — the scanner state behind the three words