What the integrations rely on
what they rely on for CI operators for contributors
The integrations call image-garden and read what it prints: allocate --spread and its one ADDRESS line, discard by address, the working directory's .image-garden/ and .image-garden.mk, the --query user keys, four environment variables, and, for the action, make targets, rebase and the snap's settings.
On the map: built on it
▶ Watch this deck — 7 scenes, ~5 min
Worth seeing first
- Projects built on image-garden — the five projects this deck is about
01 Five projects, one small interface
The integrations only run image-garden and read its output. The craft components' tests replace it with a 30-line stand-in that answers allocate --spread, writes a .run that answers --query, and accepts discard.
None of the five projects built on image-garden contains a line of its code. They run it and read what it prints. That makes the boundary small, and worth knowing exactly. The craft components' own tests show it best: they replace image-garden with a thirty-line stand-in. It answers allocate --spread with an address line, writes a .run script that reports a user and a password, and accepts discard. That's enough for the whole build script to run. Anything inside image-garden can change freely. Change one of these few things, and a CI job somewhere stops working. This deck lists them.
Details and sources
Details
- The stand-in "records how it is called … and allocates a VM that is only the fake ssh next to it":
allocatemust be called with--spread, prints<ADDRESS localhost:5555>(or<FATAL …>and exit 213 withFAKE_ALLOCATE=fail), and writes.image-garden/SYSTEM.run, which printssystem.users.0.username=ubuntuandsystem.users.0.password=ubuntu;discarddoes nothing (snapcraft-gitlab-ci testdata/bin/image-garden:6-30). It also logsPKGPROXY_URL,QEMU_MEM_OPTIONandQEMU_SMP_OPTION, which the tests check. - The five projects: snapcraft-, rockcraft-, charmcraft- and spread-gitlab-ci, and image-garden-action. spread-gitlab-ci calls nothing itself: your
spread.yaml's garden backend runs allocate and discard (sp-adhoc). - The full list, with who relies on what, is in this deck's reference below.
Read more
02 allocate's one line, and discard
allocate --spread prints progress on stderr and one line on stdout, <ADDRESS localhost:PORT> or <FATAL …> with exit 213. SSH is forwarded on 127.0.0.1. discard takes only localhost:PORT.
The heart of it is allocate, with --spread. Everything it says while it makes and starts the machine goes to standard error. Standard output gets exactly one line: ADDRESS, in angle brackets, with localhost, a colon and a port, and only once the guest's SSH server has answered. The craft scripts cut that line out with a pattern, take the port after the colon, and connect to localhost, because the port is forwarded on the loopback address. On failure, the line says FATAL instead, and the exit code is two hundred and thirteen. Later, discard takes that same address back. It refuses anything but localhost, colon, port.
Details and sources
Details
--spreadselects spread's protocol (image-garden:1023-1026). The make that prepares the system writes to stderr (>&2, image-garden:1063), as do the progress messages.- The port is forwarded with
hostfwd=tcp:127.0.0.1:PORT-:22(image-garden:1104-1107), so whatever connects must run on the same host or in the same container. - allocate waits for the
SSH-banner for up toGARDEN_SSH_WAIT_TIMEOUTseconds (600), then prints<ADDRESS localhost:PORT>; after the timeout it prints the address anyway. A machine that dies first, or a port probe that fails, gives<FATAL …>and exit 213, as does a QEMU that does not start (image-garden:1167-1216). - The craft scripts parse
sed -n -E 's/^<ADDRESS (.*)>$/\1/p'and take${address##*:}as the port; without an address line the job fails with "image-garden printed no address" (snapcraft-vm:370-373). discard localhost:PORTkills the QEMU of that pid file and removes it; any other form exits 1;discard -astops every machine of the user (image-garden:1221-1268). spread's garden backend calls it withexecand$SPREAD_SYSTEM_ADDRESS(spread.yaml:59).- The slide's port 5123 is an example; allocate picks one between 5000 and 9999.
Read more
03 The directory you run it in
image-garden works in the current directory: .image-garden/ there holds the image, the .run script and the logs, and .image-garden.mk there is included. The craft scripts and the action rely on those paths.
The second part is where things land. image-garden works in the directory you run it from. If that directory has a .image-garden directory, the image, the .run script and the logs go in there, named after the system, and the logs after the port too. And a .image-garden.mk next to it is read as part of the rules. The craft scripts lean on all three: they make a scratch directory with .image-garden inside, copy your .image-garden.mk beside it, and later collect the logs from there. The GitHub action caches that same directory, and uploads its logs when a job fails.
Details and sources
Details
GARDEN_PROJECT_DIR="$(pwd)"(image-garden:7);mk/999-local.mkhas-include $(GARDEN_PROJECT_DIR)/.image-garden.mk(999-local.mk:5).- allocate makes
SYSTEM.runandSYSTEM.qcow2in.image-garden/when it exists, otherwise in the current directory, and locksSYSTEM.lockthere (image-garden:1057-1068); it starts./SYSTEM.runfrom there, withSYSTEM.PORT.serial.log,.stdout.logand.stderr.logbeside it (image-garden:1149-1160). - The craft scripts create
$work/.image-garden, copy theimage-garden-mkinput to$work/.image-garden.mk, run allocate and discard from$work, read$work/.image-garden/SYSTEM.run --queryand copy$work/.image-garden/*.log(snapcraft-vm:337-349, snapcraft-vm:282-286, snapcraft-vm:374). - The action caches
SUBDIR/.image-gardenwithout*.log, keyed on the hash ofSUBDIR/.image-garden.mk, and uploads.image-garden/*.logwhen the job fails (action.yaml:116-123, action.yaml:480-489).
04 Two keys and four variables
The craft scripts read system.users.0.username and .password from --query, and set QEMU_MEM_OPTION, QEMU_SMP_OPTION and PKGPROXY_URL; the runner or the action sets GARDEN_DL_PKGPROXY.
Then the names. The .run script answers --query with plain key equals value lines, and the craft scripts read two keys: system.users.0.username, and system.users.0.password. Rename either, and they can't log in. On the way in, four environment variables carry settings. QEMU_MEM_OPTION and QEMU_SMP_OPTION size the machine; the craft components set them from their memory and cpus inputs. PKGPROXY_URL names the proxy the guests use, and GARDEN_DL_PKGPROXY the one that image-garden downloads images through. The action sets that last one too.
Details and sources
Details
SYSTEM.run --queryprintsvm.memory_mb=N, fromQEMU_MEM_OPTIONor the default, and the users assystem.users.N.username=andsystem.users.N.password=lines, then exits (010-qemu.mk:57-92). allocate itself readsvm.memory_mbfor its memory check.- The craft scripts read
system.users.0.usernameandsystem.users.0.password(snapcraft-vm:374-377). - The
.runscript usesQEMU_MEM_OPTIONandQEMU_SMP_OPTIONwhen set, word-split into QEMU's arguments (010-qemu-x86_64.mk:73-98); the craft scripts export-m MEMORYand-smp CPUS(snapcraft-vm:362-368). PKGPROXY_URL(http://host[:port]) is read when the image is made (010-qemu.mk:20-29).GARDEN_DL_PKGPROXYis read by the download helpers; at v0.8.0 it defaults toPKGPROXY_URL, or the snap's pkgproxy, and set but empty turns it off (image-garden:31-58). The action setsGARDEN_DL_PKGPROXY=http://127.0.0.1:3142(action.yaml:391-393).- The craft components' README asks for pkgproxy 0.12 or later for the image downloads (snapcraft-gitlab-ci README.md:213-214).
for contributors
In the code
- The
--querypreamble isEMIT_QEMU_RUN_SCRIPT_QUERY_PREAMBLE; rulegen.py passesroot distrofor cloud images,mk/050-ubuntu-core.mkpassesonly:ubuntu(rulegen.py:127, 050-ubuntu-core.mk:25). A new user model must keep user 0 able to log in with a password.
05 What the GitHub action adds
The action relies on image-garden make of SYSTEM.ARCH .qcow2 .run .user-data .meta-data .seed.iso, rebase SYSTEM.ARCH.qcow2, the pkgproxy and spread-variant options, the pkgproxy-sockets plug and the image-garden.spread app.
The GitHub action uses the snap, so its part of the contract is wider. It calls image-garden make with five files named by system and architecture: the qcow2 disk, the .run script, the user-data, the meta-data and the seed image. It runs rebase on the qcow2 file, by its file name, which rebase takes as well as the system name. It sets two snap options, pkgproxy and spread-variant, connects the snap's pkgproxy-sockets plug, and runs the snap's spread under an alias. And it expects the downloads in the snap's cache directory, which it creates before the first make.
Details and sources
Details
image-garden make --debug --dry-run SYSTEM.$(uname -m).qcow2, thenimage-garden makeof.qcow2,.run,.user-data,.meta-dataand.seed.iso; withcache-prepared-images,image-garden rebase SYSTEM.$(uname -m).qcow2; it first creates~/snap/image-garden/common/cache/dl(action.yaml:411-442).snap connect image-garden:pkgproxy-sockets pkgproxy:sockets,snap set image-garden pkgproxy=snaporpkgproxy=http://HOST-IP:3142,snap alias image-garden.spread spread,snap set image-garden spread-variant=…(action.yaml:376-401).- In the snap: the
spreadapp runsspread-chooser(snapcraft.yaml:211-224); thepkgproxy-socketscontent plug (snapcraft.yaml:254-266); the configure hook acceptspkgproxyas component, snap, none or a URL (configure:11-36) and installs the spread component forspread-variant(configure:83-101). - rebase finds the image in
.image-garden/, takes the system name or the.qcow2file name, and rewrites its backing path withqemu-img rebase -u(image-garden:943-980). The usage text, man page and README writerebase SYSTEM.ARCH; the action's.qcow2form works the same.
Read more
06 What would break them
Renaming or reshaping any of these breaks an integration, and so can a change in behaviour: allocate's use of nc once cost ten minutes per VM on aarch64 runners. The GitLab components pin a container tag; the action follows a snap channel.
So what breaks them? Changing any of those names, lines or paths, and also changing how allocate behaves underneath. That has happened. allocate once waited for SSH with nc, and on arm sixty-four runners, inside a privileged container, the host's AppArmor profile broke nc: starting a VM took ten minutes, for an SSH server that was up in seconds. A release without nc fixed it, and the craft components moved their images to it. That's the other half of the contract. The GitLab components pin an image-garden container by its tag, so a change reaches them when they update. The action installs the snap from a channel, so a release reaches its next job.
Details and sources
Details
- "The component's image is built on image-garden v0.7.2, which fixes timeouts on arm64. On aarch64 runners, such as Raspberry Pi 5 machines with Ubuntu 26.04, starting the VM no longer takes ten minutes waiting for SSH that was up in seconds: image-garden's allocate no longer uses nc, which the host's AppArmor profile breaks in a privileged container." (snapcraft-gitlab-ci NEWS:10-16). rockcraft- and charmcraft-gitlab-ci carry the same entry.
- At the commit in garden.commit, allocate probes ports and the SSH banner with bash's
/dev/tcp(tcp_probe, image-garden:785-801); v0.7.1 usednc -wfor both. - The GitLab components' images are
FROM registry.gitlab.com/zygoon/image-garden:v0.7.2(snapcraft-gitlab-ci Dockerfile:9), and each component's defaultimageis its own image of the same version, so a new image-garden reaches a pipeline when it moves to a new component release. - Known discrepancy: spread-gitlab-ci builds on an older image-garden. It is
FROM image-garden:v0.7.0(spread-gitlab-ci Dockerfile:20), before the nc fix; none of the four uses v0.8.0 yet. - The action installs
image-gardenfromimage-garden-channel,latest/stableby default, in every job (action.yaml:12-14, action.yaml:366-375).
for contributors
Before changing image-garden
- Run a change to allocate, discard,
--query, the.image-garden/layout or the variables against the craft components' tests: their stand-in records exactly what they expect (snapcraft-gitlab-ci testdata/bin/image-garden), and the action's integration workflow runs real jobs (image-garden-action .github/workflows/integration-test.yaml). - The full list is in this deck's reference.
07 Where to go from here
Next: allocate, the adhoc backend, the craft components, the GitHub action, --query and rebase.
So the contract is short: allocate with --spread and its one line, discard by address, the working directory's .image-garden, the query keys, four variables, and for the action, a few make targets, rebase and the snap's settings. To see how allocate arrives at its line, take the allocate deck, and the adhoc backend for how spread calls it. The craft deck and the action deck show the two biggest users. And the query and rebase decks cover the last two pieces in full.
Details and sources
Read more
Basis of the statements on this page
Read in the integrations at the commits in integrations.commits and in image-garden at the commit in garden.commit ("code"). Which project relies on which item was read in their scripts and templates; that a change would break them is derived. Nothing was run for this page.
What each integration relies on
| Item | Where image-garden does it | craft components | spread-gitlab-ci | image-garden-action |
|---|---|---|---|---|
image-garden allocate --spread SYSTEM.ARCH | image-garden:996-1219 | called directly | through your spread.yaml | through your spread.yaml |
one stdout line <ADDRESS localhost:PORT>, progress on stderr | image-garden:1063, :1203 | parsed with sed | spread reads it | spread reads it |
<FATAL …> and exit 213 | image-garden:1181-1213 | the job fails with the line | spread reports it | spread reports it |
SSH forwarded on 127.0.0.1:PORT | image-garden:1107 | ssh -p PORT USER@localhost | spread connects there | spread connects there |
image-garden discard localhost:PORT | image-garden:1249-1268 | in the exit trap | the discard hook, with exec | the discard hook, with exec |
state in .image-garden/ of the current directory | image-garden:1058-1068, :1149-1162 | the scratch directory | your project | cached and uploaded |
.image-garden.mk in the current directory | image-garden:7, mk/999-local.mk:5 | the image-garden-mk input | your project | its hash keys the image cache |
SYSTEM.run --query: system.users.0.username, .password | mk/010-qemu.mk:57-92 | the login | no (credentials in spread.yaml) | no |
QEMU_MEM_OPTION, QEMU_SMP_OPTION | mk/010-qemu-x86_64.mk:73-74 and the other arches | the memory and cpus inputs | your environment | no |
PKGPROXY_URL | mk/010-qemu.mk:20-29 | exported from pkgproxy-url | the runner's environment | no (snap set image-garden pkgproxy=) |
GARDEN_DL_PKGPROXY | image-garden:31-58 | the runner's environment | the runner's environment | http://127.0.0.1:3142 |
image-garden make SYSTEM.ARCH.{qcow2,run,user-data,meta-data,seed.iso} | the generated rules | no | no | yes |
image-garden rebase SYSTEM.ARCH.qcow2 | image-garden:943-980 | no | no | with cache-prepared-images |
snap: pkgproxy, spread-variant, pkgproxy-sockets, image-garden.spread | snap/hooks/configure, snap/snapcraft.yaml | no | no | yes |
~/snap/image-garden/common/cache/dl | image-garden:20-29 | no | no | created before make |
How each integration gets image-garden
| Integration | image-garden it uses | How a new release arrives |
|---|---|---|
snapcraft-gitlab-ci 91b5e29 | container v0.7.2 | a new component release |
rockcraft-gitlab-ci e8fd725 | container v0.7.2 | a new component release |
charmcraft-gitlab-ci 6fc8543 | container v0.7.2 | a new component release |
spread-gitlab-ci 3d522d4 | container v0.7.0 | a new component release |
image-garden-action 2891c7a | snap, image-garden-channel (latest/stable) | the next job |
Known discrepancies that touch the contract
- spread-gitlab-ci builds on an older image-garden: v0.7.0, before allocate stopped using
nc.
The README, image-garden's own spread.yaml and spread-demo/spread.yaml run exec image-garden in both hooks, so the <ADDRESS> and <FATAL> lines are the hook's own output and image-garden's exit status is the hook's (spread.yaml:45-59). rebase takes SYSTEM.ARCH and SYSTEM.ARCH.qcow2 alike; the action passes the second.
Roads from here
RUNTIME · make rules and the CLI
- image-garden allocate — how allocate arrives at the ADDRESS line
- Asking a .run script about its system — the --query output in full
- image-garden rebase — rebase, which the action calls on each restored image
SPREAD · tests
- The adhoc backend: allocate and discard — the same allocate and discard, called by spread
BUILT ON IT · CI integrations
- Building snaps, rocks and charms in a VM — the craft script that relies on most of this
- image-garden-action for GitHub — the action that relies on the snap's side