How image-garden.mk is put together

three parts, one file for contributors

The top-level Makefile joins data/000-systems.mk, the sorted mk/*.mk and the sorted data/050-*.gen.mk into one image-garden.mk, installed where the image-garden script looks for it; the metadata leads so that define-instance works, and the sort order decides which assignment wins.

On the map: build time

▶ Watch this deck — 7 scenes, ~6 min

Worth seeing first

01 One makefile, built from many

ONE FILE AT RUN TIME $ image-garden make ubuntu-cloud-24.04.x86_64.run make -f …/include/image-garden.mk the source tree mk/: 14 makefiles by hand data/: 13 INI files rulegen.py $ make image-garden.mk, 15,889 lines $ make install /usr/local/include/image-garden.mk
image-garden make runs make -f /usr/local/include/image-garden.mk. The source tree has 14 hand-written makefiles in mk/, 13 INI files in data/ and rulegen.py. Plain make joins them into image-garden.mk, 15,889 lines, and make install copies it to the include directory the script reads

image-garden make runs make -f PREFIX/include/image-garden.mk. The file is built by the top-level Makefile from mk/*.mk, data/*.ini and rulegen.py, and installed with make install.

When you type image-garden make, the script runs GNU make on a single makefile, image-garden.mk, from the include directory of the prefix it was installed to. Every rule for every system is in that one file. The source tree doesn't contain it. It has fourteen makefiles written by hand in the mk directory, thirteen INI files in data, and rulegen.py. Running plain make at the top of the tree joins them into one file, about fifteen thousand nine hundred lines long, and make install copies it into place. This deck shows how the parts are joined, why their order matters, and what happens when one of them fails.

Details and sources

Details

  • The script sets GARDEN_INCLUDE=@INCLUDEDIR@, which make install fills in with $(includedir), and image-garden make runs make --warn-undefined-variables GARDEN_ITSELF=… GARDEN_PROJECT_DIR=… -f "$GARDEN_INCLUDE"/image-garden.mk -C DIR "$@" (image-garden:10-12, image-garden:982-993).
  • The default goal of the top-level Makefile is all:: image-garden.mk; prefix defaults to /usr/local and includedir to $(prefix)/include (Makefile:6-21).
  • mk/ holds 14 .mk files (and a README), data/ 13 INI files (and a README). The joined file had 15,889 lines when built in a scratch copy of the pinned commit: 847 of metadata, 1,907 written by hand and 13,135 generated.
  • mk/README.md:5-10: the files "are all concatenated and installed as image-garden.mk" to /usr/local/include or /usr/include.
  • make install replaces @PREFIX@, @INCLUDEDIR@ and @DOCDIR@ in the installed script (Makefile:33-38). Run from the source tree, the script still has the placeholder, notices it, and reads image-garden.mk from its own directory, the top of the tree (image-garden:13-19).

Read more

02 Three parts, one cat

THE RECIPE cat data/000-systems.mk $(sort mk/*.mk) $(sort data/050-*.gen.mk) >image-garden.mk data/000-systems.mk 1 every system: group, release, arch, boot mk/000-garden.mk mk/010-qemu-aarch64.mk … -x86_64.mk (6) mk/010-qemu.mk mk/020-instances.mk mk/050-amazonlinux-2.mk mk/050-ubuntu-core.mk mk/050-ubuntu.mk mk/050.gentoo.mk mk/999-local.mk 2 written by hand, sorted by name settings, helpers, QEMU per architecture, named instances, hand-written systems includes your project's .image-garden.mk data/050-almalinux.gen.mk … data/050-ubuntu.gen.mk (13) 3 generated by rulegen.py, sorted data/ubuntu.ini changed: 050-ubuntu.gen.mk, 000-systems.mk rulegen.py changed: every generated part
The recipe: cat data/000-systems.mk, the sorted mk/*.mk and the sorted data/050-*.gen.mk into image-garden.mk. Part one is the metadata of every system; part two the fourteen hand-written makefiles, from 000-garden.mk to 999-local.mk, which includes the project's .image-garden.mk; part three the thirteen generated rule files. A changed INI file remakes its rule file and the metadata; a changed rulegen.py remakes them all

cat data/000-systems.mk $(sort mk/*.mk) $(sort data/050-*.gen.mk) >image-garden.mk. Each rule file depends on its INI file and on rulegen.py; the metadata depends on all of them.

The recipe that builds it is a single cat. First comes a metadata file that lists every system, with its group, release, architecture and boot type. Then the hand-written makefiles from mk, sorted by name: settings and helpers, the QEMU commands for each architecture, named instances, the rules written by hand, and last, the small file that pulls in your project's own makefile. Then the generated rule files, one per INI file, sorted too. The Makefile knows what each part is made from. Change an INI file, and its rule file and the metadata are made again. Change rulegen.py, and every one of them is. All of them are build output, which git ignores.

Details and sources

Details

  • image-garden.mk: $(wildcard mk/*.mk) $(foreach n,$(wildcard data/*.ini),data/050-$(basename $(notdir $n)).gen.mk) data/000-systems.mk with the recipe cat data/000-systems.mk $(sort $(filter mk/%.mk,$^)) $(sort $(filter data/050-%.gen.mk,$^)) >$@, then touch --reference= the newest prerequisite, so the file gets the time of its newest part rather than the time it was joined (Makefile:48-50).
  • data/050-%.gen.mk: rulegen.py data/%.ini (Makefile:57-58); data/000-systems.mk depends on every INI file and on rulegen.py (Makefile:60-64).
  • The order on the slide is the one make image-garden.mk printed in a scratch copy of the tag. The six per-architecture files are 010-qemu-aarch64, -loongarch64, -ppc64, -riscv64, -s390x and -x86_64.
  • mk/999-local.mk is one line: -include $(GARDEN_PROJECT_DIR)/.image-garden.mk (999-local.mk:4-5).
  • data/*.mk and image-garden.mk are in .gitignore:16-17; make clean removes all three kinds (Makefile:23-27).

Read more

03 Why the metadata comes first

READ TOP TO BOTTOM data/000-systems.mk mk/000-garden.mk … 050.gentoo.mk mk/999-local.mk data/050-*.gen.mk make reads image-garden.mk in this order .image-garden.mk, your project's $(eval $(call define-instance,ubuntu-cloud-24.04,webserver)) _ubuntu-cloud-24.04.x86_64_group := ubuntu _ubuntu-cloud-24.04.x86_64_group_upper := UBUNTU _ubuntu-cloud-24.04.x86_64_release := 24.04 _ubuntu-cloud-24.04.x86_64_arch := x86_64 _ubuntu-cloud-24.04.x86_64_boot := efi $ ./rulegen.py --systems-meta data/000-systems.mk data/ubuntu.ini >/dev/null once per INI file Ubuntu Core has no INI file, so no entry
make reads image-garden.mk top to bottom: data/000-systems.mk, the hand-written mk files, mk/999-local.mk, which includes the project's .image-garden.mk, and only then the generated rules. A define-instance call in .image-garden.mk for ubuntu-cloud-24.04 and webserver runs at once and looks up _ubuntu-cloud-24.04.x86_64_group, _release, _boot and the others in the metadata, which rulegen.py --systems-meta wrote. Ubuntu Core has no entry

rulegen.py --systems-meta data/000-systems.mk writes _NAME_group, _group_upper, _release, _arch and _boot. define-instance in .image-garden.mk reads them before the generated rules are read.

Why does the metadata come first, before anything else? Because of your project's makefile. It's included at the end of the hand-written part, before make has read a single generated rule. And that's where you'd call define-instance, to make, say, a web server instance of ubuntu-cloud-24.04.x86_64. make runs that macro on the spot, and it needs the base system's group, release and boot type right then. So the Makefile runs rulegen.py a second time over every INI file, with the systems-meta option, and collects five variables per system into the file that leads. Ubuntu Core has no INI file, so it has no entry there.

Details and sources

Details

  • data/000-systems.mk is removed, then built by running ./rulegen.py --systems-meta $@ $$f >/dev/null for each INI file in sorted order (Makefile:60-64). For each system rulegen.py appends _NAME_group, _NAME_group_upper, _NAME_release, _NAME_arch and _NAME_boot with := (rulegen.py:824-833). It is 847 lines for 139 systems.
  • The project file is included by the last hand-written makefile (999-local.mk:4-5), and the generated rule files are joined after it.
  • define-instance is used as $(eval $(call define-instance,BASE,NAME)); it tells a name with an architecture from one without by asking whether _BASE_group is defined, then reads the group, upper-case group, release, boot type and architecture of the base from these variables; the architecture picks the QEMU rules, so a base named .ppc64el gets the ppc64 ones (020-instances.mk:4-19, 020-instances.mk:29-56).
  • Ubuntu Core's systems are added to GARDEN_SYSTEMS by 050-ubuntu-core.mk:4-16 but get no metadata. The README says so: instances can be made of the systems in data/*.ini, "which are all but Ubuntu Core"; define-instance with an Ubuntu Core system lists the instance in list-systems, but make has no rule for it (README.md:976-979, 020-instances.mk:9-10).

Read more

04 The sort order decides

ORDER DECIDES a later plain assignment replaces an earlier one - (2D) sorts before . (2E) mk/010-qemu-x86_64.mk QEMU_SMP_X86_64 = 1 mk/010-qemu.mk QEMU_SMP_$(shell echo $(GARDEN_HOST_ARCH) | tr a-z- A-Z_) = 4 later: wins mk/010-qemu.mk EXTRA_CLOUD_INIT_APT_CONFIG = mk/050-ubuntu.mk define EXTRA_CLOUD_INIT_APT_CONFIG … later: wins mk/050-amazonlinux-2.mk mk/050-ubuntu-core.mk mk/050-ubuntu.mk mk/050.gentoo.mk a dot: after every 050- file a new file: its number says what it comes after
A later plain assignment replaces an earlier one, and make's sort compares bytes, so a dash (2D) sorts before a dot (2E). mk/010-qemu-x86_64.mk sets QEMU_SMP_X86_64 = 1 and mk/010-qemu.mk, later, sets the host architecture's to 4. mk/010-qemu.mk sets an empty EXTRA_CLOUD_INIT_APT_CONFIG and mk/050-ubuntu.mk, later, defines it. mk/050.gentoo.mk sorts after the 050- files. A new file's number says what it comes after

$(sort) is byte order: 010-qemu-x86_64.mk before 010-qemu.mk, which sets the host architecture's SMP to 4; 050-ubuntu.mk after 010-qemu.mk, so its EXTRA_CLOUD_INIT_APT_CONFIG wins; 050.gentoo.mk after the 050- files.

Sorting by name sounds harmless, but in make, order decides. A plain assignment that comes later replaces an earlier one. And make's sort compares bytes, so a dash comes before a dot. That puts the six architecture files before the common QEMU file, and it's how your machine's own architecture gets four CPUs, overriding the one its file sets. The Ubuntu file comes after the common one too, so its apt hook for old releases replaces the empty default. Gentoo's file has a dot where the others have a dash, so it sorts after them. When you add a file, pick its number for what it has to come after.

Details and sources

Details

  • The recipe sorts the mk/ files and the generated files with make's $(sort) (Makefile:48-50), which orders words lexically by byte; - is 0x2D and . is 0x2E. The order make image-garden.mk printed in a scratch copy of the pinned commit: 010-qemu-aarch64 … 010-qemu-x86_64, then 010-qemu.mk; 050-amazonlinux-2.mk, 050-ubuntu-core.mk, 050-ubuntu.mk, 050.gentoo.mk, 999-local.mk.
  • QEMU_SMP_X86_64 = 1 (010-qemu-x86_64.mk:7-7); QEMU_SMP_$(shell echo $(GARDEN_HOST_ARCH) | tr a-z- A-Z_) = 4 (010-qemu.mk:5-6), where GARDEN_HOST_ARCH is uname -m with ppc64le mapped to ppc64 (000-garden.mk:16). Both are recursive (=) assignments; the last one read wins, and a project's .image-garden.mk, read later still, can set its own.
  • EXTRA_CLOUD_INIT_APT_CONFIG = (010-qemu.mk:47-47), called by the base template (010-qemu.mk:198-198); redefined to call UBUNTU_OLD_RELEASES_APT_CONFIG in 050-ubuntu.mk:26-28. Were the files the other way round, the empty assignment would win and old Ubuntu releases would lose their apt block (derived).
  • mk/050.gentoo.mk only defines GENTOO_CLOUD_INIT_USER_DATA_TEMPLATE (050.gentoo.mk:4-8), so its place in the order changes nothing.
  • Templates, macros and target-specific variables are expanded when they are used, so for them the order matters only between two definitions of the same name (derived from make's rules for recursive variables).
  • The README gives every architecture "4 native, 1 cross-arch", which is what this order produces, a ppc64le host included (README.md:389-396, README.md:890-895).

Read more

05 Installed where the script looks

WHERE THE SCRIPT LOOKS GARDEN_INCLUDE=@INCLUDEDIR@ the script runs make -f $GARDEN_INCLUDE/image-garden.mk $ make install /usr/local/include/image-garden.mk and @INCLUDEDIR@ filled in, in the installed script the snap: make with prefix=/usr layout: bind-file /usr/include/image-garden.mk, inside the snap the container: make install /usr/local/include/image-garden.mk $ ./image-garden make … ./image-garden.mk, next to the script @PREFIX@ never replaced: the script's own directory edited mk/ or data/? run make again
The script reads image-garden.mk from GARDEN_INCLUDE, @INCLUDEDIR@. make install puts it in /usr/local/include by default and writes that directory into the script; the snap builds with prefix=/usr and a layout binds /usr/include/image-garden.mk; the container runs make install into /usr/local/include; a script run from the source tree reads image-garden.mk from its own directory. After an edit in mk/ or data/, run make again

make install puts image-garden.mk in $(includedir) and writes the prefix into the script. The snap uses prefix=/usr and a layout for /usr/include/image-garden.mk; the container installs to /usr/local; an uninstalled script reads the file next to itself.

Every way of installing image-garden puts the file where the script will look for it. make install copies it into the include directory and writes the prefix into the installed script, /usr/local unless you say otherwise. The snap builds with the prefix set to /usr, and a layout makes its copy appear at /usr/include inside the snap. The container runs the same make install, into /usr/local. And if you run the script straight from a source checkout, it notices the prefix was never filled in and reads image-garden.mk from its own directory. So after you edit anything in mk or data, run make again.

Details and sources

Details

  • make install depends on image-garden.mk, installs the script to $(bindir) and runs sed on the installed copy to write in $(prefix), $(includedir) and $(datarootdir)/doc/image-garden, and installs image-garden.mk to $(includedir), by default /usr/local/include; a different includedir or datarootdir is honoured (Makefile:6-14, Makefile:29-46).
  • The script: GARDEN_INCLUDE=@INCLUDEDIR@; when the prefix is still the placeholder, GARDEN_INCLUDE becomes the script's own directory (image-garden:10-19).
  • The snap's image-garden part passes prefix=/usr to make (snapcraft.yaml:301-303), and its layout binds /usr/include/image-garden.mk to $SNAP/usr/include/image-garden.mk (snapcraft.yaml:250-252).
  • The container copies the tree to /src/image-garden and runs make, make check and make install there (Dockerfile:69-71).
  • image-garden.mk lists the mk/*.mk files, the generated rule files and the metadata as prerequisites, so make in the tree rebuilds it after an edit; an installed copy is replaced only by make install (derived).

06 When a part fails

A FAILED PART IS DELETED 1 $ make data/gentoo.ini: error: Image 'gentoo-cloud.x86_64' has Boot='uefi', use one of platform, efi, bios data/050-gentoo.gen.mk: 0 bytes make: *** Deleting file 'data/050-gentoo.gen.mk' .DELETE_ON_ERROR in the top-level Makefile 2 $ make data/gentoo.ini: error: Image 'gentoo-cloud.x86_64' has Boot='uefi', … the same error again, not an image-garden.mk without Gentoo Boot=efi 3 $ make image-garden.mk, every system
Boot=uefi in data/gentoo.ini. The first make stops with rulegen.py's message, which names the file, the section and the values to use; the shell had created an empty data/050-gentoo.gen.mk, and make deletes it because the top-level Makefile has .DELETE_ON_ERROR. A second make without a fix fails the same way. After fixing the line, make builds the rule file, the metadata and a complete image-garden.mk

A failed ./rulegen.py data/gentoo.ini >data/050-gentoo.gen.mk leaves an empty file, and make deletes it: the top-level Makefile has .DELETE_ON_ERROR. Every later make fails the same way until the INI file is fixed.

Say you mistype a boot type in Gentoo's INI file and run make. rulegen.py stops with an error that names the file, the section and the values you can use, and make stops too. But the shell had already created Gentoo's rule file, empty, before rulegen.py ran. The top-level Makefile tells make to delete any target whose recipe fails, so make removes that file and says so. Run make again without a fix and you get the same error, not a quiet image-garden.mk without Gentoo. Fix the line, run make, and the rule file, the metadata and the joined file are all made again.

Details and sources

Details

  • Run in a scratch copy of the pinned commit with Boot=uefi in the first section of data/gentoo.ini: make stopped with data/gentoo.ini: error: Image 'gentoo-cloud.x86_64' has Boot='uefi', use one of platform, efi, bios, then make: *** [Makefile:58: data/050-gentoo.gen.mk] Error 1 and make: *** Deleting file 'data/050-gentoo.gen.mk'. A second make printed the same three lines. After restoring Boot=efi, make built the rule file, data/000-systems.mk and image-garden.mk with Gentoo's systems.
  • The rule file is written by the shell redirection >$@ before rulegen.py runs (Makefile:57-58); the metadata recipe starts with rm -f $@ and appends file by file (Makefile:60-64). .DELETE_ON_ERROR in the top-level Makefile, "so that the next make does not use it", makes make delete either of them when its recipe fails (Makefile:16-18). image-garden.mk sets the same for the rules it runs at run time (000-garden.mk:4-7).
  • The message names the section and the values the architecture accepts (rulegen.py:300-333); it is printed as FILE: error: MESSAGE without a traceback (rulegen.py:753-757).
  • An image-garden.mk already built from the earlier, good INI file is not touched by the failed run; it stays out of date, so the next make tries again (derived from the rule at Makefile:48-50).
  • make clean removes image-garden.mk, data/050-*.gen.mk and data/000-systems.mk (Makefile:23-27); it is not needed after such an error.

Read more

07 Where to go from here

ROADS image-garden.mk What rulegen.py emits the generated part Rules written by hand Ubuntu Core and the quirks Extending the first boot .image-garden.mk The SMP default order decides, in full image-garden make how it calls make A source build make and make install
Six roads from this deck: what rulegen.py emits, rules written by hand, extending with .image-garden.mk, the SMP default, the image-garden make command, and the source build

Next: rulegen.py, the hand-written rules, .image-garden.mk, the SMP default, image-garden make, and the source build.

So image-garden.mk is three parts joined in a fixed order: the metadata, the hand-written makefiles, and the generated rules. The rulegen deck explains the generated part, and the hand-written rules deck the files that stand in for it, such as Ubuntu Core. Your project's own makefile slots in between, and the deck on extending the first boot shows what to put there. The architectures deck has the CPU count example in full. The command line deck shows how image-garden make calls make, and the source build deck how to install what you've built.

Details and sources

Details

  • The recipe in one line: cat data/000-systems.mk $(sort mk/*.mk) $(sort data/050-*.gen.mk) >image-garden.mk (Makefile:48-50).

Read more

Basis of the statements on this page

Read in the top-level Makefile, rulegen.py, mk/*.mk, mk/README.md, the image-garden script, snap/snapcraft.yaml, the Dockerfile and .gitignore of image-garden at the pinned commit ("code"). The order of the parts, the line counts and the runs after an error come from make image-garden.mk in a scratch copy of that commit ("run"). What would happen with another order, and when an installed copy is replaced, are reasoned from make's rules ("derived").

The order of image-garden.mk

#FileLinesWhat it holds
1data/000-systems.mk847five variables per system from the INI files
2mk/000-garden.mk87.DELETE_ON_ERROR, cache directories, CURL, XORRISO, log banners, GARDEN_SYMLINK, help, list-systems
3mk/010-qemu-aarch64.mk218QEMU command lines, run scripts and firmware for aarch64
4mk/010-qemu-loongarch64.mk119the same for loongarch64
5mk/010-qemu-ppc64.mk72the same for ppc64
6mk/010-qemu-riscv64.mk121the same for riscv64
7mk/010-qemu-s390x.mk72the same for s390x
8mk/010-qemu-x86_64.mk334the same for x86_64, with Secure Boot and swtpm
9mk/010-qemu.mk347common QEMU settings, pkgproxy, the cloud-init templates and picker, the seed ISO, the locked unpack and convert
10mk/020-instances.mk157define-instance
11mk/050-amazonlinux-2.mk11Amazon Linux 2's template and the aarch64 CPU
12mk/050-ubuntu-core.mk328every Ubuntu Core system
13mk/050-ubuntu.mk28the apt block for old Ubuntu releases
14mk/050.gentoo.mk8Gentoo's template
15mk/999-local.mk5-include $(GARDEN_PROJECT_DIR)/.image-garden.mk
16-28data/050-almalinux.gen.mk … data/050-ubuntu.gen.mk13,135the generated rules, one file per INI file

Total: 15,889 lines. The mk/ files are sorted with make's $(sort), byte by byte, so 010-qemu-*.mk precede 010-qemu.mk and 050.gentoo.mk follows the 050- files (Makefile:48-50).

Targets of the top-level Makefile

TargetWhat it does
all (default)builds image-garden.mk
installinstalls image-garden, image-garden.mk, the manual page and the documentation under DESTDIR and prefix (or bindir, includedir, datarootdir, man1dir), and writes the prefix, the include directory and the documentation directory into the script
cleanremoves image-garden.mk, data/050-*.gen.mk and data/000-systems.mk
checkcheck-sh, check-py, check-rulegen, check-dl-pkgproxy, check-allocate, check-rebase, check-dl-unpack-lock, check-pkgproxy-guest-shell
fmtshfmt and black
distimage-garden-VERSION.tar.gz with the sources, not the generated files

Source: Makefile:16-141.

Where each install puts it

InstallPrefiximage-garden.mkSource
make install/usr/local, or prefix=$(includedir)/image-garden.mk, by default PREFIX/includeMakefile:6-46
the snap/usr/usr/include/image-garden.mk, bound by a layoutsnapcraft.yaml:250-252, :301-303
the container/usr/local/usr/local/include/image-garden.mkDockerfile:69-71
not installednoneimage-garden.mk next to the scriptimage-garden:13-19

After an error

A failed rulegen.py run leaves an empty data/050-X.gen.mk, and a failed metadata loop a partial data/000-systems.mk; the top-level Makefile has .DELETE_ON_ERROR, so make deletes the file and prints make: *** Deleting file '…' (Makefile:16-18). Every later make fails the same way until the INI file is fixed (run); make clean is not needed.

Roads from here

ARCHITECTURES · machines and firmware

RUNTIME · make rules and the CLI

FIRST BOOT · cloud-init changes

NATIVE · packages

BUILD TIME · data and rules