Changing what cloud-init does
.image-garden.mk, EXTRA_* for users for contributors
A project's .image-garden.mk, next to its tests, is read after image-garden's own make files and changes the cloud-init user-data of every image the project makes. EXTRA_CLOUD_INIT_FILES and EXTRA_CLOUD_INIT_APT_CONFIG are the simplest hooks, slow stable setup belongs here rather than in spread's prepare, and an edit rebuilds the images.
On the map: first boot
▶ Watch this deck — 7 scenes, ~5 min
Part of journeys
- Your own systems with .image-garden.mk, step 1 of 4 · next: How a user-data template is chosen
01 A file next to your tests
A .image-garden.mk in your project changes the cloud-init user-data of the images image-garden makes for it.
The images image-garden makes are deliberately plain: a distribution's cloud image, two users and open SSH. Your tests usually want a bit more, maybe a package or two, a configuration file or an apt setting. You could add those in every test's prepare step, but then every run pays for them again. Instead, put a file called .image-garden.mk next to your tests. image-garden reads it whenever it makes images for your project, and what you write there changes the cloud-init user-data of those images. This deck shows where the file goes, the two simplest hooks, and what an edit costs.
Details and sources
Details
- The README section "Local cloud-init Templates" describes the file: project-specific dependencies, cloud-init per distribution, release or instance, named instances and first-boot commands (README.md:949-960).
mk/999-local.mkincludes it:-include $(GARDEN_PROJECT_DIR)/.image-garden.mk(999-local.mk:4-5).- The plain image's user-data is the common template: users
rootand the distribution user, SSH password logins, power off at the end (010-qemu.mk:187-247).
Read more
02 Where the file goes and when it is read
image-garden runs make with your project directory; the last of image-garden's own make files includes .image-garden.mk, after image-garden's settings and before the generated system rules.
The file lives in your project directory, the one you run image-garden from. image-garden passes that directory to make, and the very last of its own make files, the local one, includes your file if it's there and quietly skips it if not. Order matters. Your file comes after image-garden's own settings, so anything you define replaces theirs, and before the generated system rules, which use what you defined. It's plain make, so you could do a lot in it. The README's advice is to keep to cloud-init templates and hooks.
Details and sources
Details
GARDEN_PROJECT_DIRis the directory image-garden runs in (image-garden:7);image-garden makepasses it to make with-f image-garden.mk(image-garden:982-993).mk/999-local.mkis-include $(GARDEN_PROJECT_DIR)/.image-garden.mk; the leading-makes a missing file no error (999-local.mk:4-5).image-garden.mkisdata/000-systems.mk, then the sortedmk/*.mk(so999-local.mklast), then the generateddata/050-*.gen.mk(Makefile:48-50). A variable defined in your file therefore replaces one defined inmk/, and is defined before the system rules expand it.- The README: "While powerful enough for arbitrary build rules, it's strongly recommended to use only cloud-init user-data templates" (README.md:1061-1070).
Read more
03 Two hooks: files and apt
EXTRA_CLOUD_INIT_FILES adds to write_files; EXTRA_CLOUD_INIT_APT_CONFIG adds a top-level section such as apt:. Both get the group as $1 and the release as $2.
The two simplest hooks are variables image-garden calls while it writes the user-data. EXTRA_CLOUD_INIT_FILES lands inside the write_files list, so it adds files to the image. EXTRA_CLOUD_INIT_APT_CONFIG lands at the top level, so it can add an apt section. Both are called with the group and the release, such as ubuntu and 24.04, so one definition can tell systems apart. Define one with define and endef, write YAML inside, indented the way cloud-init expects, and the next make puts it into every user-data.
Details and sources
Details
mk/010-qemu.mkdeclares both hooks (010-qemu.mk:47-48). The common template calls$(call EXTRA_CLOUD_INIT_APT_CONFIG,$1,$(value 3))at the top level, right afterssh_pwauth(010-qemu.mk:197-198), and$(call EXTRA_CLOUD_INIT_FILES,$1,$(value 3))at the end ofwrite_files(010-qemu.mk:236-237). In the template$1is the group and$(value 3)the release, so inside a hook they arrive as$1and$2.EXTRA_CLOUD_INIT_FILESstarts empty.EXTRA_CLOUD_INIT_APT_CONFIGis already used by image-garden for old Ubuntu releases: see the next slide.- The file
/etc/my-test.confand its content are an illustrative example, not taken from image-garden and not run for this map.
Read more
- cloud-init's own documentation describes
write_filesandapt.
04 A hook you define replaces image-garden's
image-garden defines EXTRA_CLOUD_INIT_APT_CONFIG for end-of-life Ubuntu releases. Its own .image-garden.mk redefines it and calls UBUNTU_OLD_RELEASES_APT_CONFIG inside, keeping both.
One catch: defining a hook replaces it, it doesn't add to it. And image-garden already uses the apt hook itself. For Ubuntu releases past their end of life, it points apt at the old releases archive. Redefine the hook carelessly and those releases lose their mirror. image-garden's own file shows the way. It turns off recommended packages, and inside the same apt section it calls the old releases configuration it replaced, so both survive. When you redefine a hook, keep what was there.
Details and sources
Details
mk/050-ubuntu.mklists the end-of-life releases that needold-releases.ubuntu.com(050-ubuntu.mk:12), definesUBUNTU_OLD_RELEASES_APT_CONFIG(050-ubuntu.mk:15-24) and setsEXTRA_CLOUD_INIT_APT_CONFIGto call it (050-ubuntu.mk:26-28).- image-garden's own
.image-garden.mkredefines the hook:APT::Install-Recommends "false"and the old-releases configuration with its leadingapt:removed by$(subst apt:,,…), so there is oneapt:map (.image-garden.mk:5-14). Its comment: QEMU with its recommends meant 319 packages on Ubuntu 26.04 instead of 81. - That the later definition replaces the earlier one is make's rule for variables; see where the file goes.
Read more
05 cloud-init or spread's prepare
Slow, stable setup goes in cloud-init: it runs once, when make builds the image. What depends on the code under test stays in prepare:, which runs on every fresh -snapshot machine.
So what belongs in this file, and what in spread's prepare? Think about how a test machine starts: allocate boots a fresh snapshot of the prepared image, every time. Whatever prepare or prepare-each installs is paid again on every run and on every machine. cloud-init here runs once, in the provisioning boot, and every allocate starts with it done. So put slow, stable setup in cloud-init: packages, toolchains, services, boot settings. Keep in prepare what depends on the code under test, like building your project, and per-test state. image-garden's own tests do exactly that.
Details and sources
Details
allocatestarts the.runscript with-snapshotunlessQEMU_SNAPSHOT_OPTIONsays otherwise, so each machine starts from the prepared image and its changes are thrown away (image-garden:1153-1163).- The provisioning boot, where cloud-init runs, is part of making
X.qcow2(rulegen.py:593-610); it runs again only when the image is rebuilt. - image-garden's own project splits the work this way: its
.image-garden.mkinstallspython3,make,whois,xorriso,curl,qemu-utilsand QEMU for a nested VM through cloud-init (.image-garden.mk:43-58), while itsspread.yamltop-levelprepare:builds and installs image-garden itself (spread.yaml:73-77). - Spread behaviour, not image-garden's: spread runs a project's
prepare:on each machine it allocates, a suite'sprepare:before that suite, andprepare-each:before every task. With fresh machines per run, all of them are paid on every run. - The trade-off: an edit to
.image-garden.mkrebuilds the images (next slide). For the cost over many runs drawn as two timelines, see paid for once.
Read more
06 An edit rebuilds the images
Each .user-data depends on .image-garden.mk. An edit remakes the user-data, the seed and the prepared disk of every system in the project; downloads are kept.
That's the trade-off. Every system's user-data depends on your file. Change it, and make sees each user-data is older than the file it came from, so it writes the user-data again, then the seed, then the prepared disk, which means another provisioning boot. That happens for every system your project makes, not just the one you meant, because each user-data depends on the whole file. The downloaded cloud images stay as they are. So edit it when you mean to, and expect one slow make afterwards.
Details and sources
Details
- The generated rule for each system:
X.user-data: $(MAKEFILE_LIST) $(wildcard $(GARDEN_PROJECT_DIR)/.image-garden.mk)(rulegen.py:467-471); named instances have the same dependency (020-instances.mk:75-76). Because the file is included, it is also in$(MAKEFILE_LIST), which the meta-data and.runrules depend on. X.qcow2depends on the download,X.seed.isoand the firmware copies, and its recipe creates the overlay and runs the provisioning boot (rulegen.py:593-610); the seed is made from the user-data and meta-data (010-qemu.mk:268-279).- The downloaded image and its symlink in the cache do not depend on
.image-garden.mk, so an edit downloads nothing. - The system names on the slide are examples of a project that uses three systems.
Read more
07 Where to go from here
Next: how a template is chosen, named instances, and several systems from one base image.
So: one file next to your tests, included after image-garden's own make files, with two hooks for files and apt, slow setup baked in once instead of in prepare, and a rebuild whenever the file changes. Hooks are small. For anything bigger, a template replaces the whole user-data of a group, a release or an instance, and the next deck shows how one is chosen. Named instances give you more systems from one download. And the profiles deck puts it all together.
Details and sources
Roads from here
RUNTIME · make rules and the CLI
- Named instances — more systems from one download
SPREAD · tests
- The adhoc backend: allocate and discard — the spread backend whose prepare runs on every machine
FIRST BOOT · cloud-init changes
- How a user-data template is chosen — how a whole user-data template is chosen
- Several systems from one base image — several systems from one base image, built once
- Provisioning runs once — why the provisioning boot runs only once
- The NoCloud seed — how the user-data reaches the image