Several systems from one base image
profiles you pay for once for users for CI operators
With define-instance and a cloud-init template in .image-garden.mk, one downloaded image becomes several prepared systems, such as openSUSE booted with AppArmor instead of SELinux, or a machine with heavy software already installed; the work is done once by make, not by every test run.
On the map: first boot
▶ Watch this deck — 9 scenes, ~6 min
Worth seeing first
- From a system's name to a running VM — the overlay and the one provisioning boot that a profile adds to
Part of journeys
- Your own systems with .image-garden.mk, step 4 of 4 · previous: Named instances
01 Install it once, not on every run
A project's .image-garden.mk can define extra systems from one base image, each with its own cloud-init profile, so their setup is done once instead of in every test run.
Say your tests need openSUSE twice: once as it ships, with SELinux, and once booted with AppArmor instead. Another suite needs a compiler, a database and a nested QEMU, which take minutes to install. You could do all of that in each test's prepare step, and pay for it on every run, on every machine. image-garden lets you do it once instead. A few lines in a file called .image-garden.mk describe extra systems built from the same base image, each with its own cloud-init profile. This deck builds two of them and shows when the work is done, and when it's done again.
Details and sources
Details
image-garden makeincludes the project's.image-garden.mkthrough-include $(GARDEN_PROJECT_DIR)/.image-garden.mk(999-local.mk:5). The README lists what it is for: project dependencies, cloud-init per distribution, release or named instance, named instances, and extra first-boot commands (README.md:949-965).- The README's own case for it: pre-installed dependencies speed up test iteration and reduce network use, and changes invalidate existing images (README.md:1053-1060).
- The openSUSE and SELinux/AppArmor pairing is this map's example; image-garden's source has no such profile.
Read more
02 A named instance shares the download
$(eval $(call define-instance,opensuse-cloud-16.0,apparmor)) makes opensuse-cloud-16.0@apparmor.x86_64: its own user-data, seed, overlay and .run, on the base system's download.
Start with the base system, opensuse-cloud-16.0 on x86. One line in the .image-garden.mk file, a call to define-instance with the base name and the word apparmor, creates a new system. Its name is the base name with at apparmor before the architecture. It gets its own user-data, its own seed, its own overlay and its own .run script. What it doesn't get is its own download: the overlay sits on the same verified openSUSE image as the base system. So two systems cost one download, and each keeps its own changes on top.
Details and sources
Details
define-instancetakes a base system, with or without the architecture, and an instance name; without an architecture it usesGARDEN_ARCH(020-instances.mk:4-28). The result for this example isopensuse-cloud-16.0@apparmor.x86_64.- The instance's overlay is created on the base system's download,
$(GARDEN_DL_DIR)/<group>/<base>.qcow2, withqemu-img create -f qcow2 -b(020-instances.mk:96-98). - It inherits the base system's group, release and boot type from
data/000-systems.mk(020-instances.mk:47-51). The base used here is[opensuse-cloud-16.0.x86_64](opensuse.ini:50-55). - The base must be a system defined in
data/*.ini, which is every system but Ubuntu Core: its systems are made by rules of their own and have no instances.define-instancewith an Ubuntu Core system lists the instance inlist-systems, but make has no rule for it (020-instances.mk:9-10, README.md:976-979). - The hostname follows
base-release-instance, for exampleubuntu-24-04-webserver(README.md:988-989).
Read more
03 The instance gets its own template
The most specific template wins: OPENSUSE_16.0@apparmor_CLOUD_INIT_USER_DATA_TEMPLATE first, CLOUD_INIT_USER_DATA_TEMPLATE last. A template starts with $(CLOUD_INIT_USER_DATA_TEMPLATE) and appends to its runcmd:.
Next, the profile itself. image-garden picks a user-data template by name, most specific first. For our instance it looks for a variable that joins the group in capitals, the release and the instance, followed by the words cloud-init user-data template. If that's missing, it tries the group with the instance, then the group with the release, then the group alone, and finally the common template. Start your template by expanding the common one, so you keep the users, the SSH settings and the power-off. The common template ends with a runcmd list, so the lines you add after it are more commands to run.
Details and sources
Details
- The picker tries, in order,
GROUP_RELEASE@INSTANCE_,GROUP@INSTANCE_,GROUP_RELEASE_,GROUP_, each followed byCLOUD_INIT_USER_DATA_TEMPLATE, thenCLOUD_INIT_USER_DATA_TEMPLATEitself (010-qemu.mk:293-298). For an instance the group is passed in capitals and the instance name as given (020-instances.mk:75), so the first name for this example isOPENSUSE_16.0@apparmor_CLOUD_INIT_USER_DATA_TEMPLATE. The README documents the same order (README.md:1046-1051). CLOUD_INIT_USER_DATA_TEMPLATEis the common template,BASE_CLOUD_INIT_USER_DATA_TEMPLATE(010-qemu.mk:249). It sets the users, SSH andpower_state, and ends withruncmd:(010-qemu.mk:187-247).- The README asks templates to include the common one and says it ends with
runcmd, so commands can be appended as a YAML list (README.md:1061-1070). - Appended commands run after the common template's last command,
touch /etc/cloud/cloud-init.disabled(010-qemu.mk:246). That file stops cloud-init on later boots, not the current one.
Read more
04 Changing how the system boots
Boot settings changed during provisioning apply from the next boot, the first one your tests see. The base system keeps the distribution's defaults.
In the AppArmor profile, the commands install the AppArmor packages and change the kernel command line, so the kernel starts AppArmor rather than SELinux. They run during the provisioning boot, while SELinux is still in charge. That's fine: the kernel picks its security module when it starts, so the new setting takes effect on the next boot. And the next boot comes straight after: cloud-init powers the machine off, make is done, and the first time you or your tests start this system, it comes up with AppArmor. The base system, built from the same image, still boots with SELinux.
Details and sources
Details
- The three commands on the slide are illustrative, not taken from image-garden and not run for this map. Check the AppArmor pattern name and the bootloader tool against the openSUSE release you use.
- cloud-init powers the machine off once it is done (
power_state, 010-qemu.mk:190-196), so a kernel command line written during provisioning is first used by the next boot, the first boot of the prepared image. - image-garden's own example of a boot-time change is in the spread demo: Alpine's template sets
GRUB_TIMEOUT=0and regenerates the grub configuration (.image-garden.mk:6-14).
Read more
05 Heavy software, already installed
A packages: list in a template installs software during the provisioning boot. image-garden's own .image-garden.mk installs QEMU and firmware for nested VMs, without recommends.
Software works the same way. A packages list after the common template installs whatever your tests need: compilers, a database, or, as image-garden's own tests do, QEMU and its firmware for a VM inside the VM. image-garden's own file also turns off recommended packages, because QEMU with its recommends meant 319 packages on Ubuntu 26.04 instead of 81. The spread demo has a smaller example: an mc instance of Ubuntu 24.04 that only adds Midnight Commander. Whatever the list, the packages are fetched and installed during the provisioning boot, and they end up in the prepared image.
Details and sources
Details
- image-garden's own
.image-garden.mkturns off recommended packages in anEXTRA_CLOUD_INIT_APT_CONFIG; its comment counts 319 packages with recommends and 81 without, for QEMU on Ubuntu 26.04 (.image-garden.mk:5-14). - Its Ubuntu template installs python3, make, whois, xorriso, curl, qemu-utils, a
qemu-system-*for the guest's architecture and its UEFI firmware, for nested VMs (.image-garden.mk:33-55); Debian uses the same template (.image-garden.mk:58). - The spread demo defines
ubuntu-cloud-24.04@mcand installsmcin it (.image-garden.mk:16-23). - On Debian and Ubuntu, packages come through pkgproxy when one is configured, because its apt setup runs in bootcmd; see what pkgproxy adds to user-data.
Read more
06 Paid for once, used by every run
The provisioning boot pays the install cost once, at make time. Each allocate starts from the prepared overlay with -snapshot, so runs neither repeat the install nor change the image.
Here's what that buys you. Without a profile, every test run boots the plain image and installs the same packages again, on every machine, before the first test can start. With a profile, the cost moves to make. The provisioning boot installs everything once, and the result is the prepared overlay. After that, image-garden allocate starts each machine from that overlay in snapshot mode, so whatever a test changes is thrown away, and the next run starts from the same prepared state. Ten runs or a hundred, and the install happened once.
Details and sources
Details
- The instance's
.qcow2rule creates the overlay and runs the provisioning boot once; make does not run it again while the overlay is newer than its inputs (020-instances.mk:96-103). allocateruns the.runscript with-snapshotunlessQEMU_SNAPSHOT_OPTIONsays otherwise, so writes go to a temporary file and the prepared overlay is not changed (image-garden:1153-1163).- The three-runs drawing is a schematic, not a measurement.
Read more
07 When the image is built again
Editing .image-garden.mk makes every system's user-data, seed and overlay out of date. The download is kept; the provisioning boot runs again.
Once doesn't mean never. The user-data of every system depends on your .image-garden.mk file. Edit the file, and make sees that the user-data is older. It writes it again, packs a new seed, and builds a new overlay with a new provisioning boot. The verified download stays where it is, so only the provisioning is paid again. Because every system in the project depends on the same file, a change rebuilds all of the images you've made there, not only the one whose template you touched. That's also why CI jobs that cache prepared images restore the file's modification time.
Details and sources
Details
- An instance's user-data depends on
$(MAKEFILE_LIST)and$(wildcard $(GARDEN_PROJECT_DIR)/.image-garden.mk)(020-instances.mk:75-76); the generated rules give every other system the same dependency (rulegen.py:467-468). - The seed ISO is made from the user-data (010-qemu.mk:268-279), and the overlay depends on the seed ISO (020-instances.mk:96), so a newer
.image-garden.mkreaches the overlay and its provisioning boot. - The download is a separate target, not affected by the file.
- The GitHub action restores the file's modification time with
git-restore-mtimefor this reason; see how the action caches.
Read more
08 Reaching the instance from spread
In spread.yaml the instance is opensuse-cloud-16.0-at-apparmor; allocate maps -at- to @ and adds the architecture. The credentials are the base system's, opensuse/opensuse, and --query lists them.
spread can run tests on these systems too, with one catch. spread system names can't carry an at sign, so the instance is written with -at- in spread.yaml: opensuse-cloud-16.0-at-apparmor. The allocate script of the garden backend turns -at- back into an at sign, adds the architecture, and asks image-garden for that system. The user name and password are the base system's, because the profile starts from the common template, and the instance's .run script lists them under --query, like any other. Your tests then run on both openSUSE systems, side by side, from one download.
Details and sources
Details
- The README asks for
+to be written-plus-and@to be written-at-in spread system names, and shows the allocate lines that map them back (README.md:720-732). image-garden's ownspread.yamldoes the same and adds the architecture (spread.yaml:45-47). - The instance has the base system's users,
root/rootand the group's user. Its launcher's--querylists them, as a base system's does: the rule passesroot distroand the group (020-instances.mk:89) to the preamble that printssystem.users.0andsystem.users.1(010-qemu.mk:80-88). spread itself still reads them from spread.yaml.
Read more
09 Where to go from here
Next: named instances, the template lookup order, the simpler hooks, spread names, allocate and the provisioning boot.
So a profile is a few lines of make and cloud-init that turn one download into as many prepared systems as your tests need, each paid for once. From here, the named instances deck shows everything define-instance creates. The template deck has the full lookup order, and the deck on your own changes covers the simpler hooks for files and apt settings. To run these systems under spread, the naming deck explains dash plus and dash at. And allocate, with the once-only provisioning boot, is where the saved time shows up.
Details and sources
Details
- A profile in one line:
define-instancefor the system, a template named after it that starts with$(CLOUD_INIT_USER_DATA_TEMPLATE), and make pays the provisioning once.
Read more
Basis of the statements on this page
The mechanics, define-instance, the template lookup and the rebuild rules, were read in the image-garden source at the commit this map describes ("code"). The template names for the examples follow from the lookup rule ("derived"). The openSUSE AppArmor commands are illustrative: they are not part of image-garden and were not run for this map.
A complete .image-garden.mk
Put it next to your tests, where you run image-garden make or spread.
# Two systems from one openSUSE download: the base system, # opensuse-cloud-16.0.x86_64, boots with SELinux as it ships; # this instance boots with AppArmor. $(eval $(call define-instance,opensuse-cloud-16.0,apparmor)) # Illustrative: check the pattern and the bootloader tool for your release. define OPENSUSE_16.0@apparmor_CLOUD_INIT_USER_DATA_TEMPLATE $(CLOUD_INIT_USER_DATA_TEMPLATE) - zypper --non-interactive install -t pattern apparmor - sed -i 's/security=selinux selinux=1/security=apparmor/' /etc/default/grub - update-bootloader --refresh endef # Heavy software, installed once into every Ubuntu image of the project. define UBUNTU_CLOUD_INIT_USER_DATA_TEMPLATE $(CLOUD_INIT_USER_DATA_TEMPLATE) packages: - build-essential - postgresql - qemu-system-x86 endef
- The first line of each template expands the common template: the users, SSH settings, pkgproxy setup and power-off.
- The common template ends with
runcmd:, so lines starting with-right after it are more commands. packages:starts a new top-level key after the commands.- Editing this file rebuilds the user-data, seed and overlay of every system made in this directory.
The template names for these examples
| Order | Variable name | Used by |
|---|---|---|
| 1 | OPENSUSE_16.0@apparmor_CLOUD_INIT_USER_DATA_TEMPLATE | opensuse-cloud-16.0@apparmor only |
| 2 | OPENSUSE@apparmor_CLOUD_INIT_USER_DATA_TEMPLATE | every @apparmor instance of openSUSE |
| 3 | OPENSUSE_16.0_CLOUD_INIT_USER_DATA_TEMPLATE | openSUSE 16.0 without a more specific template |
| 4 | OPENSUSE_CLOUD_INIT_USER_DATA_TEMPLATE | every openSUSE system without a more specific template |
| 5 | CLOUD_INIT_USER_DATA_TEMPLATE | the common template, for everything else |
The same order with UBUNTU, UBUNTU_24.04 and an instance such as @mc applies to Ubuntu: image-garden's spread demo defines UBUNTU_24.04@mc_CLOUD_INIT_USER_DATA_TEMPLATE.
The systems in spread.yaml
| Garden name | spread name | User and password |
|---|---|---|
opensuse-cloud-16.0.x86_64 | opensuse-cloud-16.0 | opensuse / opensuse |
opensuse-cloud-16.0@apparmor.x86_64 | opensuse-cloud-16.0-at-apparmor | opensuse / opensuse |
The allocate script maps -at- back to @ and adds .$(uname -m). See names in spread. Each instance's .run script lists the same users under --query.
Which systems can have instances
Every system defined in data/*.ini. Ubuntu Core's systems are made by rules of their own and have no instances: define-instance with one lists the instance in list-systems, but make has no rule for it (README.md:976-979).
Roads from here
RUNTIME · make rules and the CLI
- Named instances — everything define-instance creates for a named instance
- image-garden allocate — how each test machine starts from the prepared overlay
SPREAD · tests
- spread names and garden names — how an @instance is written in spread.yaml, as -at-
FIRST BOOT · cloud-init changes
- How a user-data template is chosen — the full lookup order of user-data templates
- Provisioning runs once — why provisioning runs once and then cloud-init is disabled
- Changing what cloud-init does — the simpler hooks for extra files and apt settings