The .ini files in data/
META, sections, keys for contributors
Each of the thirteen data/*.ini files has a META section with Group and Variant, then one self-contained section per system, named after it, with Arch, URL or URLProgram, a checksum, Boot and Release; rulegen.py refuses a section whose name, URL or keys do not fit.
On the map: build time
▶ Watch this deck — 7 scenes, ~5 min
Worth seeing first
- From a system's name to a running VM — the path one section takes to a running VM
01 One file per distribution
data/*.ini: thirteen files, a [META] section and one section per system. [ubuntu-cloud-24.04.x86_64] has URL, SHA256SUMS, Arch, Boot and Release.
Every system image-garden knows, apart from Ubuntu Core, is written down in the data directory, in one INI file per distribution. There are thirteen of them. Each opens with a META section, then has one section per system, and the section's name is the system's name. Open ubuntu.ini and find ubuntu-cloud-24.04.x86_64. It's five lines: the image URL, the checksum list, the architecture, the boot type and the release. That's all image-garden needs to know about this system; the rest comes from rules every system shares. This deck reads such a file key by key, and shows what rulegen.py refuses.
Details and sources
Details
- The thirteen files are almalinux, alpine, amazonlinux, archlinux, centos, debian, fedora, gentoo, opencloudos, opensuse, oracle, rocky and ubuntu; they hold 139 system sections (counted from the section headers at the tag). Ubuntu's has 57 sections besides
[META]. [META]in ubuntu.ini:Group=ubuntu,Variant=cloud(data/ubuntu.ini:4-6). The running example:URL,SHA256SUMS,Arch=x86_64,Boot=efi,Release=24.04(data/ubuntu.ini:237-242).rulegen.pyreads[META]and then every other section as one system named after the section (rulegen.py:336-351); the build runs it once per file (Makefile:57-58).- Ubuntu Core's systems are made by hand-written rules in
mk/050-ubuntu-core.mk, not from an INI file; see rules written by hand. - Lines starting with
;are comments, as in the SPDX header of each file.
02 The META section
Group=ubuntu picks the download directory, the user, the template prefix and the host name. Variant=cloud is required but not used by any rule; without either, rulegen.py says [META] has no Group=.
The META section has two keys. Group names the family, here ubuntu, and it does a lot. It's the folder in the download cache where the images go, the name of the everyday user in the guest, the prefix of the cloud-init templates make looks for, and the first half of the guest's host name. Variant says what kind of image the file holds. Every file today says cloud, and every section name repeats it, but no rule reads it yet: it's there for kinds of image still to come. Leave either key out, and rulegen.py stops and tells you which one is missing.
Details and sources
Details
- rulegen.py reads both keys with
required, which fails on a missing or empty value (rulegen.py:280-285, rulegen.py:343-348). A file without[META]stops withdata/X.ini: error: no [META] section, one without a key withdata/X.ini: error: [META] has no Group=or[META] has no Variant=(run; also checked by tests/check-rulegen.sh:172-181). - The group is the download directory
$(GARDEN_DL_DIR)/GROUP(rulegen.py:447-450), the second user and the upper-case template prefix (rulegen.py:467), and with the release the guest's host name, dots and pluses turned into dashes (mk/010-qemu.mk:97, mk/010-qemu.mk:196). The user is named after the group with the group as its password (mk/010-qemu.mk:205-208). - Known discrepancy: Variant is read but never used.
Group.variantis used only byGroup.full, which nothing calls (rulegen.py:32-48).
03 The keys of a system
Name contains Arch; exactly one of URL and URLProgram; at least one checksum key; Boot that the architecture has rules for; Release.
Now a system's own section. Its name has to contain the architecture you give in Arch, or rulegen.py calls the image suspicious. Arch is one of six: x86_64, aarch64, riscv64, s390x, ppc64 and loongarch64. The URL must name the architecture too, though amd64 counts for x86_64, and arm64 for aarch64. Instead of a URL, a section may give URLProgram, a command that prints one, but never both. Then a checksum. Boot picks the firmware: efi or bios on x86_64, efi on aarch64, riscv64 and loongarch64, and on s390x and ppc64 you leave it out. Release goes into the host name and the template names.
Details and sources
Details
Archis required and must be one ofx86_64,aarch64,riscv64,s390x,ppc64,loongarch64(rulegen.py:23-29, rulegen.py:396-401), and the section name must contain it, else "Suspicious image" (rulegen.py:402-405).- A
URLmust contain the architecture;amd64is accepted for x86_64 andarm64for aarch64, else "Suspicious URL" (rulegen.py:406-416). - Exactly one of
URLandURLProgram(rulegen.py:357-362);LocalFilenamegoes only withURL,URLPatternandLocalFilenameTemplateonly withURLProgram, as a pair (rulegen.py:364-393). Gentoo's sections show theURLProgramform (data/gentoo.ini:8-14); see URLProgram. - At least one of
SHA256SUMS,SHA512SUMS,SHA256SUM(rulegen.py:420-431); see checksums. Bootis checked against the architecture:efiorbioson x86_64,efion aarch64, riscv64 and loongarch64; s390x and ppc64 boot with the machine's own firmware,platform, which is their default and their only value (rulegen.py:300-333, rulegen.py:440). The ubuntu.ini ppc64el and s390x sections have noBoot.Releasedefaults to empty, as in Gentoo and Arch Linux (rulegen.py:441). The release goes into the host name and the template names (rulegen.py:461-467).- data/README.md describes the
[META]keys andArch,Boot,Release,URLandURLProgram(data/README.md:15-45), then the checksum and file-name keys.
Read more
04 Every section stands alone
No [DEFAULT], no inheritance: every section repeats every key. [ubuntu-cloud-24.04.ppc64el] has Arch=ppc64, which its name contains.
You'll notice the files repeat themselves. Ubuntu 24.04's aarch64 section has the same checksum list, boot type and release as the x86_64 one; only the name, the URL and Arch differ. That's on purpose. No file has a DEFAULT section, so nothing is inherited, and each section reads on its own, top to bottom. To add a system, copy its neighbour and change what differs. Watch the names, though. Ubuntu's sections for 64-bit POWER end in ppc64el, the name Ubuntu uses, while Arch says ppc64. That passes, because ppc64el contains ppc64, and the system is called ppc64el everywhere.
Details and sources
Details
- Ubuntu 24.04 for x86_64 and aarch64 (data/ubuntu.ini:237-249): the same
SHA256SUMS,BootandRelease; the name, theURLandArchdiffer. - No file in
data/has a[DEFAULT]section (searched at the tag). rulegen.py walks the sections and reads each key from the section itself (rulegen.py:336-355). Python's INI reader would hand the keys of a[DEFAULT]section to every section,[META]included (derived, not tried). [ubuntu-cloud-14.04.ppc64el]hasArch=ppc64and noBoot(data/ubuntu.ini:24-27); the check is a substring test (rulegen.py:402-405). The system's name, its.runscript and its files all sayppc64el.
Read more
05 When rulegen.py says no
data/X.ini: error: Suspicious URL … for architecture x86_64, … defines no checksum …, … has no Boot=, use efi or bios for x86_64. make then deletes the unfinished data/050-X.gen.mk.
rulegen.py checks every section while the tree is built, and one bad section stops the build. Every message starts with the file's name and the word error, then says what's wrong, with no Python traceback. A section with both URL and URLProgram, or neither, says so. A name without its architecture is a suspicious image, a URL without it a suspicious URL. A missing checksum has its own message. A missing or unknown Arch or Boot names the key and the values you can use. Two sections that would save the same file collide. And make deletes the empty rules file the failed run leaves behind, so the next make starts again from your fixed file.
Details and sources
Details
- Every message on the slide was produced by running
rulegen.pyat the pinned commit on a one-section file (Python 3.14).mainturns aValueErrorfrom reading a file intoSystemExit(f"{filename}: error: {exc}"), so it prints one line and no traceback (rulegen.py:753-757). Unreadable files and INI syntax errors, such as a section or key given twice, are reported the same way (rulegen.py:338-342). - Both or neither of
URLandURLProgram(rulegen.py:358-363); suspicious image and URL (rulegen.py:402-416); no checksum (rulegen.py:426-431). - A missing or empty
ArchgivesImage 'S' has no Arch=; an unknown oneImage 'S' has Arch='x86-64', use one of x86_64, aarch64, riscv64, s390x, ppc64, loongarch64(rulegen.py:280-297, rulegen.py:396-401). Bootis checked against the architecture (rulegen.py:300-333):Image 'S' has no Boot=, use efi or bios for x86_64(oruse efi for aarch64),Image 'S' has Boot='uefi', use one of platform, efi, bios, andImage 'S' has Boot=bios, which aarch64 does not support, use efi. On s390x and ppc64 a missingBootmeansplatform. See when Boot is wrong or missing.- The file-name collision is checked after reading, for names known when the rules are written, and reported the same way:
data/X.ini: error: per-group local filename collision in group 'G': 'F' used by 'S1' and 'S2'(rulegen.py:758-775). - The build runs
./rulegen.py data/X.ini >data/050-X.gen.mk(Makefile:57-58). The shell creates the output before rulegen.py fails; the top-level Makefile has.DELETE_ON_ERROR(Makefile:16-18), so make deletes it (run:make: *** Deleting file 'data/050-example.gen.mk'). The nextmakeruns rulegen.py again. tests/check-rulegen.shchecks these messages, and that none of them comes with a traceback (tests/check-rulegen.sh:65-72, tests/check-rulegen.sh:96-223).
06 Trying a new section
./rulegen.py data/X.ini, then make, ./image-garden make list-systems and ./image-garden make SYSTEM.run, all from the tree. make check-rulegen runs tests/check-rulegen.sh.
To try a new section, you don't need to install anything. Run rulegen.py on the file and read the rules it prints for your system: the download, the overlay and the .run script. Then run make at the top of the tree. It writes that file's rules and the list of systems again, and joins image-garden.mk. The image-garden script next to it uses that file as it is. So ./image-garden make list-systems shows your system, and the same command with its .run name builds it. Last, make check-rulegen runs the project's own tests of these checks.
Details and sources
Details
rulegen.pytakes INI files as arguments and prints the rules on standard output (rulegen.py:741-775).makerebuildsdata/050-X.gen.mkfor a changeddata/X.ini,data/000-systems.mkfrom every file, and joinsimage-garden.mk(Makefile:48-64).- An uninstalled
image-gardenstill has@PREFIX@in it and then looks forimage-garden.mknext to itself (image-garden:10-19); see installing from source. list-systemsprintsGARDEN_SYSTEMS, to which every section adds itself (mk/000-garden.mk:85-87, rulegen.py:453-454).make check-rulegenrunstests/check-rulegen.sh(Makefile:99-100), which printsPASS: NAMEorFAIL: NAMEfor each case (tests/check-rulegen.sh:30-74).make checkalso runs shellcheck and mypy, see installing from source.| lessand| grepon the slide are only for reading the output.
07 Where to go from here
Next: checksums, URLProgram, boot types, what rulegen.py emits and how image-garden.mk is put together.
That's the whole format: a META section, then one self-contained section per system, checked when the tree is built. The names deck shows how those section names are put together. The checksum deck explains the three checksum keys and the URL placeholder. The URLProgram deck covers images whose address changes with every build. The boot types deck shows what Boot picks. And to see what rulegen.py makes of a section, and how its output becomes image-garden.mk, take the generator and assembly decks.
Details and sources
Read more
- System names, checksums, URLProgram, boot types.
- What rulegen.py emits, how image-garden.mk is put together.
- data/README.md, the project's own notes on the
[META]keys, the keys of an image section, checksums and file names.
Read more
Basis of the statements on this page
Read in rulegen.py, data/*.ini, data/README.md, the Makefile and tests/check-rulegen.sh at the pinned commit ("code"). The error messages were produced by running that rulegen.py on one-section files with Python 3.14, and make's on a failing build ("run"). What Python's INI reader would do with keys the files do not use is "derived".
The [META] section
| Key | Required | Example | Used for |
|---|---|---|---|
Group | yes | ubuntu | download directory $(GARDEN_DL_DIR)/GROUP, the second user and its password, the template prefix GROUP_…, the host name |
Variant | yes | cloud | nothing yet; every section name repeats it |
The keys of a system section
| Key | Required | Rule | Notes |
|---|---|---|---|
| section name | must contain Arch | becomes the system name, GROUP-VARIANT-RELEASE.ARCH by convention | |
Arch | yes | x86_64, aarch64, riscv64, s390x, ppc64, loongarch64 | the name must contain it |
URL | one of the two | must contain the architecture (amd64, arm64 accepted) | its basename is the cached file name |
URLProgram | one of the two | a shell command, run by make, that prints the URL | see URLProgram |
LocalFilename | no | only with URL; no /, not . or .. | overrides the cached file name |
URLPattern, LocalFilenameTemplate | no | only with URLProgram, together; one capture group, @1 | a stable cached file name |
SHA256SUMS, SHA512SUMS, SHA256SUM | at least one | lists by URL, or one hash; @URL@ only at the start of the first two | see checksums |
Boot | yes, except on s390x and ppc64 | efi or bios on x86_64; efi on aarch64, riscv64, loongarch64; platform (the default) on s390x and ppc64 | see boot types |
Release | no | any text, default empty | host name and per-release templates |
Sources: rulegen.py:17-29, rulegen.py:300-333, rulegen.py:336-445, data/README.md:15-88.
rulegen.py's messages
Every message is one line on standard error, data/X.ini: error: and then the text below; there is no traceback (rulegen.py:753-775). make then deletes the empty data/050-X.gen.mk (.DELETE_ON_ERROR, Makefile:16-18).
| Mistake | Message after data/X.ini: error: |
|---|---|
no [META] section | no [META] section |
no Group or Variant in [META] | [META] has no Group= or [META] has no Variant= |
| a section or key given twice, unreadable file | the INI reader's or the system's message, as in While reading from 'data/X.ini' [line 3]: option 'group' in section 'META' already exists |
both URL and URLProgram | Image 'S' defines both URL and URLProgram |
| neither | Image 'S' defines neither URL nor URLProgram |
no Arch | Image 'S' has no Arch= |
unknown Arch | Image 'S' has Arch='x86-64', use one of x86_64, aarch64, riscv64, s390x, ppc64, loongarch64 |
name without Arch | Suspicious image 'S' for architecture A |
| URL without the architecture | Suspicious URL 'U' for architecture A |
| no checksum | Image 'S' defines no checksum, use one of SHA256SUMS=, SHA512SUMS= or SHA256SUM= |
no Boot where one is needed | Image 'S' has no Boot=, use efi or bios for x86_64 (use efi for aarch64, and so on) |
unknown Boot | Image 'S' has Boot='uefi', use one of platform, efi, bios |
a Boot the architecture lacks | Image 'S' has Boot=bios, which aarch64 does not support, use efi |
@URL@ in another key | Image 'S' uses @URL@ in 'sha256sum', it is only allowed in SHA256SUMS= and SHA512SUMS= |
@URL@ not at the start, or twice | Image 'S' SHA256SUMS= must use @URL@ once, at the start of the value, e.g. @URL@.sha256 |
| two sections, one cached file name | per-group local filename collision in group 'G': 'F' used by 'S1' and 'S2' |
A new section, step by step
- Copy the section next to it in the same file and change the name,
URLandArch(or the release). ./rulegen.py data/X.iniand read the rules for the new name.make, then./image-garden make list-systemsand./image-garden make NAME.runfrom the tree.make check-rulegen, andmake checkbefore sending the change.
Roads from here
START HERE
- How systems are named — how the section names are put together
ARCHITECTURES · machines and firmware
- Boot types — what the Boot key picks, and what a missing one costs
BUILD TIME · data and rules
- Every image has a checksum — the three checksum keys and the @URL@ placeholder
- URLProgram and local file names — URLProgram, URLPattern and LocalFilenameTemplate
- What rulegen.py emits — the make rules rulegen.py writes for a section
- How image-garden.mk is put together — how the generated rules become image-garden.mk