include
pulling in other files for application developers for administrators
An include line is replaced by the contents of another file before the profile is compiled. Angle brackets search the include path and quotes name a path as written. The line takes no comma. A file is included once in the preamble and once in each profile. The hash include spelling still works.
On the map: preamble and reuse
▶ Watch this deck — 8 scenes, ~6 min
Worth seeing first
- The preamble — the preamble, where the first include line stands
Part of journeys
- Write your first profile, step 2 of 12 · previous: The preamble · next: Abstractions and tunables
01 Fourteen lines that are really eight hundred
include replaces itself with a file's contents. apparmor_parser -p prints the assembled text.
Demo's profile is fourteen lines long, and two of them say include. Before anything is compiled, each of those lines is replaced by the contents of another file, and those files include more files. Ask the parser for the assembled text and you get seven hundred ninety-nine lines. That's how hundreds of profiles share the same definitions and the same basic rules without copying them. The compiler works on that one text, and the kernel sees none of this. This deck is about that one line: how it names a file, where the file is looked for, and how often it's taken.
Details and sources
Details
- Run with the tree's parser for this deck:
-pon the 14-line example of this district, with the tree's owntunables/andabstractions/on the search path, prints 799 lines. - Under
profiles/apparmor.dof the tree there are 1126includelines; 358 of them areinclude if exists. - An include is resolved at compile time only. Nothing of it reaches the kernel: the kernel receives the compiled rules.
- On the slide the tree's
profiles/apparmor.dis drawn as/etc/apparmor.d, where it is installed. - The program
demoand its profile are invented for the map.
02 The keyword and one file name
include <magic/path> or include "path", no comma. #include is the older spelling of the same line.
The line is short: the word include, a blank, and one file name. Put the name in angle brackets and the parser searches for it. Put it in quotes and it's opened exactly as written, spaces included. You'll also meet the old spelling with a hash in front. It looks like a comment and isn't: #include is the same include. One thing trips people up: every rule in a profile ends with a comma, and an include doesn't. And the optional words if exists in the middle have a deck of their own.
Details and sources
Details
- Four spellings:
include <...>,include "...",#include <...>,#include "...". A fifth, an unquoted path, is treated like the quoted form (include_tests/ok_16.sd, a test the parser is expected to accept). - The keyword needs a blank after it. Blanks inside the angle brackets are trimmed.
<"name">is accepted. - The line takes no comma.
- The quoted path with spaces on the slide is from include_tests/ok_26.sd.
#includegives the warningdeprecated use of '#include'only with--warn=pound-includeor--warn=all(run with the tree's parser for this deck). Shipped policy has one#includeleft (locale:14).- The name may be a directory: see include if exists.
Read more
for AppArmor developers for security researchers
In the code
include/{WS}pushes the scanner stateINCLUDE(parser_lex.l:804-809).- The file name patterns of that state: parser_lex.l:376-398.
#include/.*\r?\nis matched before the comment rule (parser_lex.l:796-802, parser_lex.l:815-818).
03 Angle brackets search, quotes do not
< >: the -I directories in order, then the base directory. " ": the path as written, relative to the working directory.
Where does the parser look? For a name in angle brackets it walks a list of directories. First come any you gave with -I, in the order you gave them. Last comes the base directory, which is /etc/apparmor.d unless -b says otherwise. The first directory that has the file wins. A name in quotes gets no search at all. It's opened as written, so a relative path depends on the directory the parser happens to run in. That's why shipped policy uses angle brackets. And when nothing is found, the error says could not open, with the name as you wrote it.
Details and sources
Details
- A magic path, the name in angle brackets, is tried in each directory given with
-I, in the order given, and then in the base directory. The first directory that has the name is used. At most 100-Idirectories are accepted. - The base directory is
/etc/apparmor.d, or the directory given with-b. - A quoted path, relative or absolute, is opened as written and does not use the search path.
- The walk through two directories on the slide is derived from the code and was not run with two directories.
/srv/policyand"local/demo.rules"are invented names. - First error, run with the tree's parser for this deck:
AppArmor parser error for /work/inc-missing.sd in profile /work/inc-missing.sd at line 3: Could not open 'local/demo', exit status 1. The second error is recorded in the research notes./work/inc-missing.sdis the scratch name of the file. - Known discrepancy: apparmor_parser(8) describes
-band-Iin terms that do not match the code. The deck follows the code; the man page says-bis "for resolving #include directives defined as relative paths" (apparmor_parser.pod:172-180).
for security researchers
Attack surface
- The first directory that holds the name wins, so a directory given with
-Ishadows files of the base directory for every magic path. Whoever chooses the parser's options, or can write to a directory on the path, chooses the text of every profile: see Who can change policy. - A quoted relative path is resolved against the working directory of whatever starts the parser.
- Both points are reasoned from the code and were not run.
for AppArmor developers for security researchers
In the code
include_filename(filename, search, if_exists)(parser_lex.l:162-225).search_path()(parser_include.c:154-187); the list of directories (parser_include.c:66-152).- A quoted name goes to
fopen()unchanged (parser_lex.l:185-186). - The options: parser_main.c:596-600.
04 Ask for the text the compiler sees
apparmor_parser -p FILE prints the text after includes. Marker lines start with ##.
When a profile misbehaves and you can't see why, look at what the compiler actually got. The option -p prints the profile after every include has been resolved. Each file is announced by a marker line that starts with two hashes. So you can see that tunables/global pulled in tunables/home, and that home defines the @{HOME} variable. More files follow, and then your own lines. Inside the profile the same happens for the abstraction. The markers come in three kinds: included, skipped as a duplicate, and failed. The next scene is about the second kind.
Details and sources
Details
- Run with the tree's parser for this deck:
-pon the example file prints 799 lines. The markers of the preamble, in order:##included <tunables/global>,<tunables/home>,<tunables/home.d>,<tunables/multiarch>,<tunables/multiarch.d>,<tunables/proc>,##failed include <tunables/proc.d>, and so on throughalias,kernelvars,system,xdg-user-dirs,share,etcandrun. The slide leaves most of them out and keeps the order. - Inside the profile:
##included <abstractions/base>,<abstractions/crypto>,<abstractions/openssl>,<abstractions/upki>, with##failed includefor their.ddirectories and for<local/demo>. - Three markers:
##included <name>,##skipped duplicate include <name>,##failed include <name>. Quoted paths print with quotes:##included "/path". -pstops after this step and compiles nothing. Other preamble lines (abi, assignments,alias) are printed unchanged, and variables are not expanded in this output.##failed includeappears only forinclude if exists; a plain include of a missing file is an error.- Known discrepancy:
-pprints##included "PATH"for a quoted include that does not exist.
Read more
for AppArmor developers for security researchers
In the code
- The markers are printed at parser_lex.l:173-177, parser_lex.l:184 and parser_lex.l:222-223.
05 A file is taken once per profile
One include of a file per preamble and per profile, child profile or hat. Later ones are skipped silently.
Abstractions include other abstractions, and profiles include several of them, so the same file is often named twice. That's harmless, because the parser keeps a list of the files it has taken. The second include of a file is skipped without a word, and -p shows it as skipped duplicate. The same goes inside a profile. The list isn't global, though. The preamble has one, and every profile, child profile and hat starts its own. So the next profile in the file gets the abstraction again, as it should.
Details and sources
Details
- Run with the tree's parser for this deck: the nine lines on the slide, as one file, with
-p. The output has##included <tunables/global>and later##skipped duplicate include <tunables/global>; insidedemo,##included <abstractions/base>and##skipped duplicate include <abstractions/base>; insidedemo-helper,##included <abstractions/base>again. Exit status 0. The research notes record the same for one profile. - The parser remembers which files it has included by the path it opened.
- A file that includes itself is stopped by the same list (include_tests/recursive_2.sd, a test the parser is expected to accept).
- Including one abstraction in a parent and in its child profile is two includes.
- A new file on the command line starts with an empty list.
- Known discrepancy: An include inside an
ifor qualifier block makes the same include in the rest of the profile a no-op.
Read more
for AppArmor developers for security researchers
In the code
g_includecacheis astd::set<std::string>(file_cache.h:27-47).- The skip by path: parser_include.c:165-170, parser_lex.l:171-172, parser_lex.l:179-181.
- A new cache per profile, restored at its closing brace (parser_yacc.y:396-397, parser_yacc.y:454-455); reset per file (parser_main.c:1267-1268).
06 The grammar never sees an include for AppArmor developers for security researchers
parser_lex.l:804-809 matches the keyword, 162-225 opens the file and pushes a buffer, 400-406 pops it.
The scanner matches the word include in its initial state and switches to a state that takes one file name. Then it calls include_filename. That function searches the path and checks the include cache. It opens the file and pushes a new input buffer on top of the current one. From then on, the tokens the grammar receives simply come from the other file. At its end the buffer is popped and the old file continues. One consequence: an include can stand anywhere a token can, even in the middle of a line, as long as the included text fits the grammar at that point.
Details and sources
for AppArmor developers for security researchers
Details
- The keyword and the state: parser_lex.l:804-809.
include_filename(): parser_lex.l:162-225. The pop at end of file: parser_lex.l:400-406.search_path(): parser_include.c:154-187. - Error messages take the file and line from the include stack (
current_filename,current_lineno, parser_include.c:197-239). That is why they readin profile <included file> at line N. - Each opened file updates the newest-source time stamp used for the cache decision (
update_mru_tstamp, parser_lex.l:154, parser_lex.l:204, parser_lex.l:210). MAX_NEST_LEVELis defined and not used (parser_include.c:58); no depth limit was found.- A conditional block that is not taken still has its includes pulled in, because the scanner knows nothing of the block. Recorded in the research notes:
if false { include <does/not/exist> }givesCould not open 'does/not/exist'. - The line in the bottom strip is from include_tests/ok_19.sd, a test the parser is expected to accept.
for security researchers
Attack surface
- What an include contributes depends on where it stands: a file of rules inside
deny { }becomes deny rules. - The include cache makes the first include of a file in a profile the only one. See the known discrepancy under "A file is taken once per profile".
for AppArmor developers for security researchers
Read more
07 Mistakes and their messages
No comma after an include. Could not open 'NAME' means the name was not found on the search path.
The classic mistake is the comma. You've typed a dozen rules, each with a comma, and you put one after the include too. The parser then sees a comma with no rule in front of it. Forget the closing bracket, or the blank after the keyword, and the parser complains about an unexpected character. A misspelt name gives could not open, with the name you wrote. The old hash spelling isn't an error, and only warns if you ask for all warnings. And a quoted relative path may work on your machine and fail when the service starts.
Details and sources
Details
- Rows 1 to 5 were run with the tree's parser for this deck, each as a line of a small profile for demo; rows 1 to 3 are also recorded in the research notes. Rows 1 to 4 exit with status 1.
- Row 1 in full:
AppArmor parser error for /work/i06a.sd in profile /work/i06a.sd at line 2: syntax error, unexpected TOK_END_OF_RULE, expecting TOK_ID or TOK_MODE or TOK_SET_VAR. The file names under/work/are scratch names of the runs. - Row 4 in full:
AppArmor parser error for /work/i06d.sd in profile /work/i06d.sd at line 3: Could not open 'abstraction/base'. - Row 5, with
--warn=all:Warning from /work/i06e.sd (/work/i06e.sd line 3): deprecated use of '#include', exit status 0. The warning class ispound-include, which is not in the default set;--warn=pound-includeturns it on alone. - Row 6 was not run: it follows from the way a quoted path is opened (derived).
- Other messages in the code, not produced here:
fstat failed for '%s',Failed to process filename.
for AppArmor developers for security researchers
In the code
- The messages: parser_lex.l:195-200, parser_lex.l:386. The warning class: parser_main.c:295; the default set: parser.h:77-78.
08 What gets included, and when it may be missing
Onward: include if exists, abstractions and tunables, running the parser, comments.
That's include: one file name, angle brackets to search, no comma, once per profile. The form with if exists, which also explains the .d directories, has a deck of its own. What the shipped files under abstractions and tunables are for is in the deck on abstractions and tunables. The parser's options for the search path are in the deck on running the parser. And why #include isn't a comment is in the comments deck.
Details and sources
Read more
Basis of the statements on this page: "tree" means the parser built from the AppArmor commit 836052d21 (6.0.0~alpha2) was run on the fragment (-Q -K --skip-kernel-load, with the tree's abi/5.0 file as kernel feature set and the tree's profiles/apparmor.d on the search path), with nothing loaded into a kernel; "code" means the parser source was read and not run; "derived" means the result was reasoned from that code. Nothing here was observed on a running kernel.
Grammar
INCLUDE: ( 'include' | '#include' ) [ 'if' 'exists' ] ( <magic/path> | "path" | path )
- Five spellings:
include <...>,include "...",#include <...>,#include "..."and a bare path, which is treated like the quoted form (code; the testinclude_tests/ok_16.sd). - A blank must follow the keyword (tree). Blanks inside the angle brackets are trimmed and
<"name">is accepted (code). - No comma. A comma after the file name is a syntax error (tree).
if existsmakes a missing file acceptable, and the name may be a directory: see syn-include-if-exists.- The line may stand in the preamble and in a profile body (tree).
How the name is resolved
| Form | Search path | Relative to | Example |
|---|---|---|---|
<name> | yes: each -I directory, then the base directory | the directory that has it | include <abstractions/base> |
"name" | no | the working directory of the parser, unless absolute | include "/etc/demo/extra.rules" |
| bare name | no | as the quoted form | include simple_tests/include_tests/includes/base |
The second and third example paths are illustrations (derived); the first is in the running example (tree).
The search path
-I DIRadds a directory; several are tried in the order given, at most 100 (code).- The base directory comes last. It is
/etc/apparmor.d, or the directory given with-b(code). - The first directory that has the name is used (code).
- apparmor_parser(8) says
-bis "for resolving #include directives defined as relative paths". The code uses it as the last directory of the search for magic paths and not for quoted relative paths. This page follows the code.
The markers of -p
apparmor_parser -p FILE prints the text after includes and compiles nothing.
| Marker | Meaning | Line in the running example |
|---|---|---|
##included <name> | the file was taken; its text follows | ##included <tunables/global> |
##skipped duplicate include <name> | already taken in this preamble or profile | none; see the scene "A file is taken once per profile" |
##failed include <name> | not found; only include if exists gets this far | ##failed include <local/demo> |
The running example gives 799 lines for 14 lines of input (tree).
Once per scope
The parser remembers which files it has included. A new, empty list starts at:
- each file named on the command line (code);
- the preamble of that file (tree);
- each profile, child profile and hat (code; tree for two profiles in one file).
A second include of a file within one list is skipped without a message. A file that includes itself is stopped the same way (the test include_tests/recursive_2.sd).
Errors and warnings
| Message | Cause | Class, default | Basis |
|---|---|---|---|
Could not open 'NAME' | the name was not found; exit status 1 | error | tree |
syntax error, unexpected TOK_END_OF_RULE, expecting TOK_ID or TOK_MODE or TOK_SET_VAR | a comma after an include in a profile | error | tree |
Lexer found unexpected character: '<' (0x3c) in state: INCLUDE | the closing bracket is missing | error | tree |
Lexer found unexpected character: '<' (0x3c) in state: INITIAL | no blank after the keyword | error | tree |
deprecated use of '#include' | the #include spelling | warning pound-include, off by default | tree |
fstat failed for 'NAME' | the opened file could not be examined | error | code |
Failed to process filename | the file name could not be taken from the line | error | code |
for AppArmor developers for security researchers
In the code
- Scanner:
include/{WS}pushesINCLUDE(parser_lex.l:804-809); the#includeform (parser_lex.l:796-802); file name patterns (parser_lex.l:376-398); end of file pops the buffer (parser_lex.l:400-406). include_filename(filename, search, if_exists)(parser_lex.l:162-225) andsearch_path()(parser_include.c:154-187).- The include stack that gives messages their file and line (parser_include.c:197-239).
- The include cache
g_includecache(file_cache.h:27-47); one per profile (parser_yacc.y:396-397, parser_yacc.y:454-455). - Each opened file updates the time stamp used for the cache decision (parser_lex.l:154).
- Known discrepancy: An include inside an
ifor qualifier block makes the same include in the rest of the profile a no-op. Recorded in the research notes. - Known discrepancy:
-pprints##included "PATH"for a quoted include that does not exist. Recorded in the research notes. - Known discrepancy: apparmor_parser(8) describes
-band-Iin terms that do not match the code (apparmor_parser.pod:172-180).
Roads from here
START HERE
- The profile language at a glance — every kind of line on one page
PREAMBLE AND REUSE
- include if exists — optional includes and whole directories
- Abstractions and tunables — the two kinds of file that shipped policy includes
- The preamble — includes above the first profile
PROFILE STRUCTURE
- Comments and line structure — why hash include is not a comment
- Quoting and escaping — quoted paths with spaces
RUNNING THE PARSER
- Running apparmor_parser (no slides yet) — the options that set the search path and print the assembled text
- The policy cache (no slides yet) — included files and the age of the cache
INSIDE THE PARSER
- The lexer — the scanner pulls the file in