Globbing (AARE)

* ** ? [] {} for application developers for security researchers

The pattern syntax of paths and names. One star stays inside a path component and two stars cross slashes. A question mark is one character. Brackets are a character set and braces are alternatives. A pattern must match the whole name.

On the map: file access

▶ Watch this deck — 9 scenes, ~7 min

Worth seeing first

Part of journeys

01 One rule, many files

GLOBBING /etc/demo/ demo.conf tls/ cert.pem key.pem conf.d/ 10-net.conf demo's configuration /etc/demo/demo.conf r, one file /etc/demo/** r, everything below AARE: AppArmor regular expressions * ** ? [ ] { , } the pattern must match the whole name, start to end
The directory /etc/demo with a configuration file and two subdirectories. A rule naming one file covers one line of the tree; a rule ending in two stars covers everything below the directory. The five pattern constructs are star, two stars, question mark, brackets and braces.

Paths in rules are AARE patterns: *, **, ?, [...], {a,b}. A pattern matches a name only if it matches all of it.

Demo reads its configuration from a directory, and you don't want to write one rule per file. You couldn't keep up anyway: files come and go. So a path in a rule is a pattern. It looks like a shell glob, with a few differences that matter for security. AppArmor calls these patterns AARE, AppArmor regular expressions. There are five constructs: one star, two stars, the question mark, brackets and braces. One rule before we start. In the kernel a pattern has to match the whole name, from the first slash to the last character. A pattern that only matches the beginning matches nothing.

Details and sources

Details

  • The pattern syntax is called AARE, AppArmor regular expressions. The manual page describes it in the section "Globbing (AARE)" (apparmor.d.pod:2463-2504).
  • The five constructs: *, **, ?, [set] or [^set], and {alt,alt}. A backslash in front of one of these characters makes it literal.
  • Whole-name matching, the kernel: aa_dfa_match walks the compiled policy over the complete, NUL-terminated name and returns the state it finished in (match.c:507-519); the permissions are those of that final state (file.c:221-231). A pattern that covers only the beginning of a name ends in a state without the permission.
  • The name the kernel matches is the absolute path of the file as the kernel builds it, with a / at the end if it is a directory (path.c:167-177).
  • The same syntax is used in rule paths, in both names of a link rule and in profile attachments.
  • To see what a pattern compiles to, run apparmor_parser -Q -K --skip-kernel-load -D rule-exprs FILE. It prints each pattern and the regular expression made from it, and loads nothing.
  • The program demo and its files are invented for the map. Nothing on this slide was observed on a running kernel.

02 The five constructs

GRAMMAR stands for can match / ? * any run of characters, also none no ** any run of characters, also none yes ? exactly one character no [abc] [a-c] [^a] one character from the set, or one not in it [^a]: yes {a,b} one of the alternatives; they may nest as written everything else is literal; a backslash makes a glob character literal: \* \? \[ \{
A table of the five pattern constructs: star, two stars, question mark, brackets and braces, with what each stands for and whether it can match a slash. Two stars and a negated set can; star and question mark cannot.

* and ? do not match /. ** does. [^a] matches any character except a, including /. {a,b} is alternation.

Here they are. One star is any run of characters inside one path component: it stops at a slash. Two stars is any run of characters including slashes, so it reaches down into subdirectories. A question mark is exactly one character, and not a slash. Brackets are one character from a set, or with a caret in front, one character that isn't in the set. Watch that one: a negated set does match a slash. Braces list alternatives separated by commas, and an alternative can be anything, even more braces. Put a backslash in front of any of these and it's an ordinary character again.

Details and sources

Details

  • What each construct matches: *: any run of characters except /, also none; **: any run of characters including /, also none; ?: exactly one character except /; [set]: exactly one character of the set, with ranges such as [a-c]; [^set]: exactly one character not in the set, and that can be /; {alt,alt}: one of the comma-separated alternatives; at least one comma.
  • "Also none" has one exception: a * or ** that makes up a whole path component must match at least one character. See the scene "The directory itself".
  • What the parser built from the AppArmor tree at 836052d21 (-Q -K --skip-kernel-load -D rule-exprs, nothing loaded into a kernel) prints for them: * becomes [^/\x00]*, ** becomes [^\x00]*, ? becomes [^/\x00], brackets are copied unchanged, {a,b} becomes (a|b).
  • Escapes: \*, \?, \[, \], \{, \} are the literal characters; \, is a literal comma inside braces; \\ is a backslash.
  • A backslash before any other character is dropped. With --warn=format (or --warn=all) the parser says Character q was quotedunnecessarily, dropped preceding quote ('\') character; the missing space is in the parser's text.
  • The manual page lists the constructs at apparmor.d.pod:2469-2504. It does not mention that a negated set matches /.

for AppArmor developers for security researchers

In the code

03 * stays in the directory, ** goes down

* ** /etc/demo/ demo.conf tls/ cert.pem key.pem conf.d/ 10-net.conf /etc/demo/ PATTERN r, four rules, one per column * yes no no no no no one level ** yes yes yes yes yes yes every level *.conf yes no no no no no this directory only **.conf yes no no no no yes any depth * and ** also match names that start with a dot
The directory /etc/demo as a tree, and four columns for the patterns star, two stars, star dot conf and two stars dot conf. Star matches only demo.conf; two stars match every file and directory below; star dot conf matches demo.conf; two stars dot conf match demo.conf and conf.d/10-net.conf.

/etc/demo/* matches one level; /etc/demo/** matches every level. Neither treats dot files specially.

Take the same directory and four rules that differ only in the last part of the pattern. A single star matches demo.conf and nothing else here. The other files are one level down, and a star doesn't cross a slash. Two stars match all of it: every file and both subdirectories. *.conf picks the files ending in .conf in this directory only. **.conf picks them at any depth, so the one inside conf.d is included. And unlike the shell, a star does match a leading dot, so hidden files are covered too.

Details and sources

Details

  • Conversions printed by the parser built from the AppArmor tree at 836052d21 (-Q -K --skip-kernel-load -D rule-exprs, nothing loaded into a kernel): /etc/demo/* becomes /etc/demo/[^/\x00][^/\x00]*; /etc/demo/** becomes /etc/demo/[^/\x00][^\x00]*; /etc/demo/*.conf becomes /etc/demo/[^/\x00]*\.conf; /etc/demo/**.conf becomes /etc/demo/[^\x00]*\.conf.
  • Neither star treats a leading dot specially: /foo/* matches /foo/.hidden. The shell's rule about hidden files does not apply.
  • /foo/* does not match /foo/a/b or the directory /foo/a/. /foo/** matches /foo/a, /foo/a/ and /foo/a/b/c.
  • A star in the middle of a component may match nothing: /foo/*bar matches /foo/bar and /foo/xbar; /a*b matches /ab and not /a/b.
  • /foo** (no slash before the stars) matches /foo, /foobar, /foo/ and /foo/a/b.
  • /**/foo matches /a/foo and /a/b/foo, and not /foo.
  • Three stars are read as ** followed by *, which matches the same names as **.
  • The match verdicts are derived from the printed regular expressions; they were not tested on a running kernel.

for AppArmor developers for security researchers

In the code

04 The directory itself

THE SLASH RULES demo opendir("/etc/demo") name: /etc/demo / the kernel: a directory's name ends in a slash /etc/demo/** r, no: ** here needs at least one character /etc/demo/ r, yes: the directory /etc/demo/{,**} r, yes: the directory and everything below /etc/demo/*/ r, the subdirectories tls/ and conf.d/ /etc//demo///x is read as /etc/demo/x
The program demo lists /etc/demo and the kernel names the directory /etc/demo/ with a trailing slash. Four rules are compared: /etc/demo/** does not match it, /etc/demo/ does, /etc/demo/{,**} matches the directory and everything below, and /etc/demo/*/ matches the subdirectories.

Directory names end in /. /etc/demo/** does not match /etc/demo/; /etc/demo/ r, or /etc/demo/{,**} r, does.

Listing a directory is a read of the directory itself, and the kernel writes a directory's name with a slash at the end. You might expect the two star rule to cover it, with the stars matching nothing. It doesn't. When stars make up a whole path component, right after a slash, they must match at least one character. That's deliberate: a rule for the contents isn't a rule for the directory. So write the directory with its slash, or use the brace idiom: an empty alternative, or two stars. */ gives you the subdirectories one level down. And doubled slashes in a pattern are collapsed for you.

Details and sources

Details

  • In the kernel the name of a directory is given a / at the end before it is matched, except for the root directory, which is already / (path.c:167-177). The manual page says the same: "Only rules that match a trailing slash will match directories" (apparmor.d.pod:2506-2529).
  • The slash rule: a * or ** that stands directly after a / and is followed by a / or by the end of the pattern must match at least one character. The source gives the purpose: /foo/* should not match /foo/, /foo/*bar should match /foo/bar, and /*/foo should not match //foo (parser_regex.c:178-205).
  • What to write: the directory itself, for listing it: /dir/ r,; the directory and everything below: /dir/{,**} r,; everything below, not the directory: /dir/** r,; the files directly in it: /dir/* r,; the directories directly in it: /dir/*/ r,.
  • The rule looks only at the character just before the star, so it does not apply inside braces: /foo/{*,bar} matches /foo/.
  • /** matches every name that starts with / except / itself; /{**,} includes /.
  • Repeated slashes in a pattern are collapsed to one before conversion: /etc//demo///x compiles as /etc/demo/x. A pattern that starts with exactly two slashes keeps them, and a trailing slash is kept.
  • The collapsing works on the text and does not look inside braces: /a/{/b,c} keeps the inner /.
  • Conversions were printed by the parser built from the AppArmor tree at 836052d21 (-Q -K --skip-kernel-load -D rule-exprs, nothing loaded into a kernel). The kernel behaviour is read from the kernel source, not observed.

Read more

for AppArmor developers for security researchers

In the code

05 ? and brackets

? [ ] /var/log/demo-?.log w, demo-1.log yes demo-a.log yes demo-10.log no demo-.log no /var/lib/demo/[a-c]* r, alpha yes cache yes data no /var/lib/demo/[^.]* r, state yes .hidden no /a[^b]c r, /axc yes /abc no /a/c yes: the slash matched to stay inside a component, list the slash: /a[^b/]c
Four rules with names that match or do not match. A question mark matches exactly one character. A set a to c matches one of those letters. A negated set of dot skips hidden files. The negated set in /a[^b]c also matches the slash in /a/c.

? is one non-slash character. [a-c] is one of a range. [^b] is any one character except b, including /; write [^b/] to exclude the slash.

A question mark is one character. Not a slash, and not zero characters: the log rule matches demo-1.log but not demo-10.log. Brackets are one character from a list or a range. [a-c]*, matches names beginning with a, b or c. With a caret the set is negated: [^.]*, is a way to skip hidden files. But be careful with negated sets. They match anything that isn't listed, and that includes the slash. /a[^b]c matches /a/c. When that matters, put the slash in the set.

Details and sources

Details

  • ? is exactly one character and never /: /a?c matches /abc, and not /ac, /a/c or /abbc.
  • [bc] is one of the listed characters, [b-d] one of a range: /a[b-d]e matches /abe, /ace and /ade.
  • [^b] is any one character except b, and / is such a character: /a[^b]c matches /axc and also /a/c. Write [^b/] to keep the match inside one path component.
  • Conversions printed by the parser built from the AppArmor tree at 836052d21 (-Q -K --skip-kernel-load -D rule-exprs, nothing loaded into a kernel): /var/log/demo-?.log becomes /var/log/demo-[^/\x00]\.log; /var/lib/demo/[a-c]* becomes /var/lib/demo/[a-c][^/\x00]*; /var/lib/demo/[^.]* becomes /var/lib/demo/[^\.][^/\x00]*.
  • Inside brackets {, } and , are literal.
  • A defect of the parser at this commit: * and ? inside brackets are converted as globs instead of being literal. /a[?]b compiles without a message to /a[[^/\x00]]b, which does not match /a?b. Write \? outside brackets for a literal question mark.
  • An empty set fails without a pattern message: /a[] r, prints only ERROR processing regexs for profile NAME, failed to load.
  • A negated set in the parser's tests: /foo[^me]bar r, (ok_inv_char_class.sd).
  • The match verdicts are derived from the printed regular expressions; they were not tested on a running kernel.

for security researchers

Attack surface

  • A negated class crosses directories: [^b] matches / unless the slash is listed, as in [^b/] (run with the parser built from the AppArmor tree at 836052d21, nothing loaded into a kernel, -D rule-exprs). * and ? stop at a slash; [^...] does not.
  • * and ? inside brackets are not literal. /a[?]b compiles to /a[[^/\x00]]b and /c[x*]d to /c[x[^/\x00]*]d, each a different set from the one written, with no message and exit status 0 (run with the parser built from the AppArmor tree at 836052d21, nothing loaded into a kernel; parser_regex.c:169-254 has no test for being inside a class, unlike the brace case at parser_regex.c:286-289). Write [\*] or [\?] for the literal character.
  • An alternation defeats the slash rule: /foo/* needs at least one character after the slash, while /foo/{*,bar} compiles to /foo/([^/\x00]*|bar) and also matches the directory /foo/ (same run).
  • Matching is done on the name the kernel builds for the object at the time of the check. Names that are disconnected or outside the task's root are handled by the attach_disconnected family of flags: other profile flags.

for AppArmor developers for security researchers

In the code

06 Braces, and variables that become braces

{ , } @{ } @{APP_DATA} = /var/lib/demo /srv/demo owner @{APP_DATA}/ {cache,state} /** rwk, variable expanded {/srv/demo,/var/lib/demo}/{cache,state}/** /srv/demo/cache/... /srv/demo/state/... /var/lib/demo/cache/... /var/lib/demo/state/... {a,b{c,d}} nesting /x{,.bak} empty alternative: /x and /x.bak /{etc/demo,srv/*}/x slashes and stars inside {cache} is an error: a group needs a comma. Up to 49 levels of nesting. \{literal\} real braces in a name
A rule with the variable APP_DATA and the brace group cache, state. The parser expands the variable into a brace group of its two values, so the one rule covers four directory trees. Below: nested braces, an empty alternative and escaped braces.

{a,b} matches a or b. A variable with several values expands to {v1,v2}. {a} and {} are errors; {a,} is allowed.

Braces are a choice: cache or state. This rule is line 26 of the example profile, and it also uses a variable. APP_DATA has two values, and the parser rewrites the variable into a brace group of its values before it does anything else with the pattern. So one line covers four directory trees. Alternatives can be nested, can be empty, and can contain slashes and stars. A group does need at least one comma, because without one there's no choice to make. And if you need a real brace in a name, put a backslash in front of it.

Details and sources

Details

  • {a,b} matches a or b. An alternative may be empty, may contain /, globs and further brace groups: /a{b,c{d,e}} matches /ab, /acd and /ace; /a{b,} matches /ab and /a.
  • A group needs at least one comma. {a} and {} are refused with Regex grouping error: Invalid number of items between {}; {a,} is accepted.
  • Groups nest up to 49 levels; the 50th is refused with Regex grouping error: Exceeded maximum nesting of {}.
  • \{ and \} are literal braces, \, a literal comma inside a group: /a{b\,c,d} matches /ab,c and /ad.
  • A variable is replaced by its values before the pattern is converted. One value is inserted as it is; several become a brace group (variable.cc:252-340).
  • With @{APP_DATA} = /var/lib/demo /srv/demo, the path @{APP_DATA}/{cache,state}/** is expanded to {/srv/demo,/var/lib/demo}/{cache,state}/** and compiled to (/srv/demo|/var/lib/demo)/(cache|state)/[^/\x00][^\x00]*. Printed by the parser built from the AppArmor tree at 836052d21 (-Q -K --skip-kernel-load -D rule-exprs, nothing loaded into a kernel); the order of the values is the parser's.
  • Glob characters in a value stay active, and a doubled slash where a value meets the rest of the path is trimmed.
  • Expansion applies to rule paths and to link targets (block_rules.cc:399-406).
  • An undeclared variable: Failed to find declaration for: @{NAME}, then ERROR expanding variables for profile NAME, failed to load.

Read more

for AppArmor developers for security researchers

In the code

07 convert_aaregex_to_pcre for AppArmor developers for security researchers

PARSER: AARE TO REGEX process_variables filter_slashes convert_aaregex_to_pcre aare_rules::add_rule convert_aaregex_to_pcre(aare, anchor=0, glob_default, pcre, &first_re_pos) AARE /etc/demo/** regex /etc/demo/ [^/\x00] parser_regex.c:180-205 [^\x00]* :207-220 * -> [^/\x00]* ** -> [^\x00]* ? -> [^/\x00] { , } -> ( | ) [...] copied; . + | ( ) ^ $ escaped returns pattern_t Basic TailGlob Regex Invalid /etc/demo/** is a TailGlob: a literal and one final ** Basic: no glob at all
The parser's steps for a path pattern: process_variables, filter_slashes, convert_aaregex_to_pcre and add_rule. The pattern /etc/demo/** is rewritten to /etc/demo/[^/\x00][^\x00]*, with a table of the other translations and the four return values Basic, TailGlob, Regex and Invalid.

convert_aaregex_to_pcre (parser/parser_regex.c:118-431) emits [^/\x00]* for *, [^\x00]* for ** and a leading [^/\x00] under the slash rule; filter_slashes (parser/parser_regex.c:61-96) runs first.

By the time convert_aaregex_to_pcre runs, variables are expanded and repeated slashes are filtered. The function walks the pattern once. At a star it looks at the last character it wrote: after a slash, and before another slash or the end, it first emits one mandatory non-slash character. That's the slash rule. Then two stars become a run of anything but NUL, and one star a run without slashes. Braces become a group with bars. Brackets are copied through for the regex parser, which is why a negated class can match a slash. The return value says basic, tail glob or regex.

Details and sources

for AppArmor developers for security researchers

Details

08 Patterns the parser refuses

COMMON MISTAKES /var/lib/demo/{cache} rw, a group needs a comma Regex grouping error: Invalid number of items between {} /var/lib/demo/{cache,state rw, the group is left open Regex grouping error: Unclosed grouping or character class, expecting close } /var/lib/demo/cache] rw, a ] with no [ before it Regex grouping error: Invalid close ], no matching open [ detected then, after any of them, with no line number: Unable to parse input line '/var/lib/demo/{cache}' ERROR processing regexs for profile demo, failed to load /etc/demo/extra, r, quote it: "/etc/demo/extra," r, syntax error, unexpected TOK_END_OF_RULE, expecting TOK_MODE an unquoted path cannot end in a comma: the comma ends the rule
Three rejected patterns with the parser's message: a brace group without a comma, a brace group that is not closed, and a closing bracket without an opening one. Each is followed by a line quoting the pattern and a line saying the profile failed to load. A fourth rule shows a path that ends in a comma and must be quoted.

Pattern errors read "Regex grouping error: ..." followed by "Unable to parse input line 'PATTERN'" and name the profile, not the line.

Three pattern mistakes you'll meet: a group with no comma, a group that's left open, and a closing bracket with no opening one. Each gets a regex grouping error. But these are found late, after the whole file has been parsed, so the message has no line number. Instead the parser quotes the pattern back at you: unable to parse input line, then the pattern, and then the profile failed to load. Search your file for the quoted text. One more looks like a pattern problem and isn't: a path can't end in a comma unless you quote it, because the comma ends the rule.

Details and sources

Details

  • Messages are from the parser built from the AppArmor tree at 836052d21 (-Q -K --skip-kernel-load --warn=all, nothing loaded into a kernel).
  • /var/lib/demo/{cache} rw,: Regex grouping error: Invalid number of items between {}. Test file: bad_re_brace_3.sd.
  • /var/lib/demo/{cache,state rw,: Regex grouping error: Unclosed grouping or character class, expecting close }. An unclosed [ gives the same text, also with "expecting close }".
  • /var/lib/demo/cache] rw,: Regex grouping error: Invalid close ], no matching open [ detected.
  • Each is followed by Unable to parse input line 'PATTERN' and ERROR processing regexs for profile NAME, failed to load, then ERROR generating policy re for profile NAME, failed to load and Errors found in file. Aborting.. The first two lines are prefixed with the path of the parser program. None has a file name or a line number.
  • A pattern that ends in a backslash: Regex error: trailing '\' escape character. This one is from the source and was not run.
  • A syntax error, by contrast, has the form AppArmor parser error for FILE in profile FILE at line N: MESSAGE. /etc/demo/extra, r, gives one: syntax error, unexpected TOK_END_OF_RULE, expecting TOK_MODE.
  • An unquoted path cannot end in a comma: /foobar, r, fails (bad_comma_1.sd) and "/foobar," r, is accepted (ok_comma_2.sd).

for AppArmor developers for security researchers

In the code

09 Where to go from here

ROADS AARE * one level · ** all levels · ? one character · [ ] a set · { , } a choice where patterns appear syn-file-rules attachment syn-link-rules what feeds them syn-variables syn-quoting what happens next fe-aare re-tree k-match conversion, tree, kernel matcher syntax-map every kind of line on one page
AARE in the centre with a recap of the five constructs, and roads to three groups of decks: where patterns appear (file rules, link rules, attachment), what feeds them (variables, quoting) and what happens next (the conversion, the expression tree, the kernel matcher), plus the syntax map.

Next: syn-variables, syn-quoting, attachment (patterns in profile heads), fe-aare and re-tree (compilation).

You'll use these five constructs in every profile you write. The same patterns appear in the head of a profile, where they decide which programs it attaches to, and in both names of a link rule. Variables and quoting feed into them: one expands into braces, the other lets a name hold a space or a comma. And if you want to know how a pattern becomes something the kernel can match in one pass over the name, follow it into the compiler decks: the conversion, the expression tree, and the kernel's matcher.

Details and sources

Details

  • * stays inside one path component; ** crosses slashes; ? is one character and not /; [set] is one character; {a,b} is a choice.
  • A pattern must match the whole name. A directory's name ends in / (the kernel, read from the source).
  • [^set] can match /.
  • apparmor_parser -Q -K --skip-kernel-load -D rule-exprs FILE prints what each pattern compiles to.
  • Commits read for this deck: AppArmor 836052d21 (6.0.0~alpha2), the kernel-rc2 (97d73c825).

Read more

Basis of the statements on this page: "tree" means the parser built from the AppArmor commit above was run on the pattern (-Q -K --skip-kernel-load --warn=all, with -D rule-exprs for the regular expressions), with nothing loaded into a kernel; "ran" means the researcher ran the pattern as recorded in the findings, without the run being repeated for this page; "code" means the parser or kernel source was read, not run. Match verdicts are derived from the regular expression the parser prints. Nothing here was observed on a running kernel.

Grammar

PATTERN:   a sequence of
  *          any run of characters except /, also none
  **         any run of characters including /, also none
  ?          exactly one character except /
  [set]      exactly one character of the set; ranges with -  ([a-c])
  [^set]     exactly one character not in the set (this can be /)
  {alt,alt}  one of the comma-separated alternatives; at least one comma
  \c         the character c, literally
  c          any other character, literally
  • The pattern language is called AARE, AppArmor regular expressions.
  • A pattern matches a name only if it matches the whole name. In mainline Linux 7.4 the kernel walks the complete name and uses the permissions of the state it ends in (code: linux:security/apparmor/match.c:518, linux:security/apparmor/file.c:226-228).
  • * and ** that make up a whole path component must match at least one character: see The slash rules.
  • *, ** and ? do not treat a leading dot specially. /foo/* matches /foo/.hidden (tree).
  • An alternative may be empty, may contain /, globs and further brace groups.
  • AARE is used in rule paths (syn-file-rules), in both names of a link rule (syn-link-rules) and in profile attachments (attachment). Values in xattrs=(...) use the same syntax with one difference: there the globs may also match a NUL byte (code: glob_null, parser/parser_regex.c:582-587). See syn-xattrs.

Translation table

What apparmor_parser -D rule-exprs prints for each construct in a path (tree; lines in parser/parser_regex.c).

AARE                             regular expression          lines
*                                [^/\x00]*                   221-226
**                               [^\x00]*                    207-220
* or ** right after / and
  followed by / or the end       [^/\x00] then the above     180-205
?                                [^/\x00]                    231-254
[ ... ]                          copied unchanged            256-280
{ , }                            ( | )                       282-357
^ $ outside [ ]                  \^ \$                       361-368
. + | ( )                        escaped with \              374-379
\* \[ \] \{ \}                   kept escaped                169-177, 256-309
\? and \, outside [ ]            ? and ,                     232-237, 333-346
\\                               \\                          152-167
\ and any other character        the character (warning)     382-403

Conversions of the patterns on the slides (tree):

PatternRegular expression
/etc/demo/*/etc/demo/[^/\x00][^/\x00]*
/etc/demo/**/etc/demo/[^/\x00][^\x00]*
/etc/demo/*.conf/etc/demo/[^/\x00]*\.conf
/etc/demo/**.conf/etc/demo/[^\x00]*\.conf
/etc/demo/{,**}a group of two alternatives: empty, or [^\x00]*
/var/log/demo-?.log/var/log/demo-[^/\x00]\.log
/var/lib/demo/[a-c]*/var/lib/demo/[a-c][^/\x00]*
/var/lib/demo/[^.]*/var/lib/demo/[^\.][^/\x00]*
/etc//demo///x/etc/demo/x

Worked matches

Verdicts are derived from the printed regular expression (tree for the conversion; /a[^b]c, /a[bc]d, /a[b-d]e and the brace rows were also run by the researcher).

PatternMatchesDoes not match
/foo/*bar/foo/bar, /foo/xbar/foo/x/bar
/foo/*/foo/a, /foo/.hidden/foo/ (the directory), /foo/a/, /foo/a/b
/foo/*/the directories directly in /foo/foo/, /foo/a
/a*b/ab, /axb/a/b
/foo/**/foo/a, /foo/a/, /foo/a/b/c/foo/, /foo
/foo**/foo, /foobar, /foo/, /foo/a/b/fo
/**/foo/a/foo, /a/b/foo/foo
/**every name that starts with // itself
/{**,}every name that starts with /, and /names that do not start with /
/a?c/abc/ac, /a/c, /abbc
/a[bc]d/abd, /acdeverything else
/a[b-d]e/abe, /ace, /adeeverything else
/a[^b]c/axc, /a/c/abc, /ac
/a{b,c{d,e}}/ab, /acd, /ace/ac
/a{b,}/ab, /a/ac
/a{b\,c,d}/ab,c, /ad/ab
/***read as ** then *: the same names as /**/ itself

The slash rules

  • A * or ** that stands directly after a / and is followed by a / or by the end of the pattern must match at least one character (code: parser/parser_regex.c:180-205). The purpose, from the comment in the source: /foo/* should not match /foo/, /*/foo should not match //foo, and /foo/*bar should still match /foo/bar.
  • The test looks only at the character written just before the star, so it applies only directly after a literal /. /foo/{*,bar} matches /foo/ (tree). /{*,x}/y matches //y, a name the kernel does not produce (code).
  • In mainline Linux 7.4 the name of a directory ends in /, except the root directory (code: linux:security/apparmor/path.c:167-177). A pattern matches a directory only if it can end in /.
To allowWrite
the directory itself (for listing it)/dir/ r,
the directory and everything below/dir/{,**} r,
everything below, not the directory/dir/** r,
the files directly in it/dir/* r,
the directories directly in it/dir/*/ r,

Repeated slashes

  • Before conversion, runs of / in a pattern are collapsed to one / (code: filter_slashes, parser/parser_regex.c:61-96). /a//b///c/ compiles as /a/b/c/ (tree).
  • A pattern that starts with exactly two slashes keeps them: //a//b compiles as //a/b; ///usr becomes /usr.
  • Trailing slashes are collapsed to one and not removed.
  • It runs after variable expansion, on rule paths, link targets and profile attachments.
  • It works on the text and does not look inside brace groups: /a/{/b,c} keeps the inner / and requires // in the name (tree).

Variables in patterns

The variable syntax is in syn-variables. For patterns:

  • A variable is replaced by its values before conversion. One value is inserted as it is; several become a brace group {v1,v2} (code: parser/variable.cc:252-340).
  • With @{APP_DATA} = /var/lib/demo /srv/demo, the path @{APP_DATA}/{cache,state}/** is expanded to {/srv/demo,/var/lib/demo}/{cache,state}/** (tree; the order of the values is the parser's).
  • At the joins a doubled slash is trimmed: with @{V}=/x/, @{V}/z becomes /x/z (tree).
  • Glob characters in a value stay active: with @{V}=a*b, /@{V} compiles as /a[^/\x00]*b (tree).
  • A value list counts as one level toward the nesting limit.
  • Expansion applies to rule paths and link targets (code: parser/block_rules.cc:399-406). A variable in an exec target is not expanded: see syn-named-transitions.
  • An undeclared variable: Failed to find declaration for: @{NAME}, then ERROR expanding variables for profile NAME, failed to load (tree).

Escapes

WrittenMeans
\* \? \[ \] \{ \}the literal character
\,a literal comma inside a brace group
\\a backslash
\ and any other characterthat character; the backslash is dropped, with a warning of class format that is off by default

Escape sequences such as \x41 are decoded earlier, by the lexer: see syn-quoting.

Limits

  • Brace groups nest up to 49 levels; the 50th is refused (tree; MAX_ALT_DEPTH is 50, parser/parser_regex.c:122).
  • A brace group needs at least one comma. {a} and {} are errors; {a,} is accepted.
  • Inside brackets {, } and , are literal.
  • An unquoted path cannot end in a comma or contain a comma followed by white space, because the comma ends the rule there. /foobar, r, fails and "/foobar," r, is accepted (parser/tst/simple_tests/file/bad_comma_1.sd, ok_comma_2.sd). /a,b r, is accepted (ran).

Errors and warnings

Syntax errors are printed as AppArmor parser error for FILE in profile FILE at line N: MESSAGE (tree). Pattern errors come from a later stage and have no line number. Each of them is followed by Unable to parse input line 'PATTERN' and ERROR processing regexs for profile NAME, failed to load, and every line is prefixed with the path of the parser program.

MessageCauseExample
Regex grouping error: Invalid number of items between {}a brace group without a comma/var/lib/demo/{cache} rw, (tree)
Regex grouping error: Unclosed grouping or character class, expecting close }a { or a [ that is not closed/var/lib/demo/{cache,state rw, (tree)
Regex grouping error: Invalid close ], no matching open [ detecteda ] without a [/var/lib/demo/cache] rw, (tree)
Regex grouping error: Invalid close }, no matching open { detecteda } without a {; printed after the "Invalid number of items" message/a} r, (ran)
Regex grouping error: Exceeded maximum nesting of {}50 nested brace groups(tree)
Regex error: trailing '\\' escape characterthe pattern ends in a backslash(code)
Character %c was quotedunnecessarily, dropped preceding quote ('\\') charactera warning, class format, off by default; the missing space is in the parser's text/a\qb r, (ran)
Failed to find declaration for: @{NAME} then ERROR expanding variables for profile NAME, failed to loadan undeclared variable@{HOME}/x r, without the tunables (ran)
syntax error, unexpected TOK_END_OF_RULE, expecting TOK_MODEan unquoted path that ends in a comma; a syntax error, with a line number/etc/demo/extra, r, (tree)

Known discrepancies

These are defects in the parser at the commit above, not intended behaviour. They are listed so that you can avoid them.

  • * and ? inside brackets are translated as globs instead of being literal. /a[?]b compiles without a message to /a[[^/\x00]]b, which matches /a[]b and not /a?b (tree). Write \? outside brackets, and [\*] for a star in a set.
  • An unclosed [ reports "expecting close }".
  • The warning text reads "quotedunnecessarily", without the space.
  • An empty set fails without a pattern message: /a[] r, prints only ERROR processing regexs for profile NAME, failed to load (tree).

for AppArmor developers for security researchers

Code map

pattern_t convert_aaregex_to_pcre(const char *aare, int anchor, int glob,
                                  std::string& pcre, int *first_re_pos)
                                                    parser/parser_regex.c:118
WhatWhere
the slash rule (mandatory [^/\x00])parser/parser_regex.c:180-205
** and the TailGlob classificationparser/parser_regex.c:207-220
*parser/parser_regex.c:221-226
?parser/parser_regex.c:231-254
[ and ]parser/parser_regex.c:256-280
{, , and }; MAX_ALT_DEPTHparser/parser_regex.c:282-357, 122
error exitsparser/parser_regex.c:405-431
filter_slashesparser/parser_regex.c:61-96
the call for a rule path and for a link targetparser/parser_regex.c:659-660, 700-701
glob_default (0, paths) and glob_null (1, xattr values)parser/parser.h:458-459
pattern_t: Basic, TailGlob, Regex, Invalidparser/immunix.h:139-144
variable::expand_by_alternationparser/variable.cc:252-340
negated sets: NotCharSetNodeparser/libapparmor_re/expr-tree.h:683-709
the character set grammarparser/libapparmor_re/parse.y:109-150
unit testsparser/parser_regex.c:1505-1516, 1576-1671
  • anchor is 0 at every call for rules.
  • Basic means the pattern has no glob. Exec conflicts use a different test, made on the expression tree (parser/libapparmor_re/aare_rules.cc:98-105): a rule is exact when it has no star, any-character or negated-set node, so braces and plain sets are exact too: see syn-exec-modes.
  • TailGlob is a literal followed by one final **; everything else with a glob is Regex.
  • The brackets are copied to the output and parsed by the regular expression grammar of the DFA compiler, which is why a negated set is not limited to one path component. The next steps are in fe-aare and re-tree.

Roads from here

START HERE

ATTACHMENT, NAMESPACES, STACKING

FILE ACCESS

PREAMBLE AND REUSE

  • Variables — variables that expand into alternatives

PROFILE STRUCTURE

IN PRACTICE · security-sensitive programs

IN THE KERNEL

INSIDE THE PARSER

THE DFA ENGINE · libapparmor_re