image-garden-action for GitHub
image-garden-action for CI operators
image-garden-action runs spread tests on GitHub runners. It installs snapd, the pkgproxy snap and the image-garden snap, opens /dev/kvm, points image-garden at pkgproxy, makes the garden system's image and runs spread on it, then uploads spread.log, artifacts and boot logs, and keeps the pkgproxy cache in the GitHub cache.
On the map: built on it
▶ Watch this deck — 7 scenes, ~5 min
Worth seeing first
- The adhoc backend: allocate and discard — the garden backend your spread.yaml needs
Part of journeys
- From your laptop to a git forge, step 5 of 6 · previous: spread-gitlab-ci · next: What pkgproxy gives a garden
01 Your spread tests on GitHub
zyga/image-garden-action installs image-garden on a GitHub runner and runs your spread tests on one garden system per job.
You've got a spread.yaml with the garden backend, and the tests pass on your machine. On GitHub you'd have to repeat what you did by hand on every runner: install the snap, get pkgproxy going, make the image, run spread, and keep the downloads for next time. The image-garden action does all of that in one step of your workflow. You tell it which garden system to use, and it runs the same spread command you run at home, on a fresh runner, every time. Let's follow one job from checkout to the logs.
Details and sources
Details
- The action "allows running full-system integration tests, using
spread, on vanilla images" prepared by image-garden (image-garden-action README.md:7-17). - It describes itself as installing image-garden and using the bundled copy of spread (image-garden-action action.yaml:5-7).
- The README calls the action pre-release and points at canonical/snapd-smoke-tests as a user (image-garden-action README.md:22-25).
Read more
02 The workflow
One job per system: actions/checkout with fetch-depth: 0, then zyga/image-garden-action@v0 with garden-system. The other inputs are optional.
Here's a whole workflow. One job per system: this one is Debian 13. The first step checks out your code, and note fetch-depth: 0. The action uses the git history to give your .image-garden.mk its real change time, so make can tell that a cached image is still current. Without the history it warns. The second step uses the action, and the one input it needs is garden system. Others are optional: spread variant picks spread plus, spread tasks runs a subset, and the cache inputs come later.
Details and sources
Details
- The README's example workflow: one job per system,
actions/checkoutwithfetch-depth: 0("essential for git restore-mtime to work correctly"), thenzyga/image-garden-action@v0withgarden-system(image-garden-action README.md:405-432). - The inputs, with defaults (image-garden-action action.yaml:8-56):
garden-systemis required;spread-variant,spread-live,spread-subdir,spread-backend(garden),spread-tasks,spread-artifacts; the cache inputscache-prepared-images,save-cache,save-package-cache,cache-scope,cache-key-prefix; and the three snap channels. - When the project has a
.image-garden.mk, the action runsgit restore-mtimeon it, so make sees when it last changed in git, not when it was checked out; without the history it warns "check out with fetch-depth: 0" (image-garden-action action.yaml:124-152).
Read more
03 What it installs on the runner
The action makes /dev/kvm usable, installs snapd and the pkgproxy snap (unless port 3142 is taken), then snap install --devmode image-garden.
The runner is an ordinary Ubuntu machine, so the action prepares it. If /dev/kvm exists, it opens it to every user, so QEMU gets hardware acceleration; without it, everything runs emulated and much slower. It installs or refreshes snapd, then the pkgproxy snap, unless something already listens on pkgproxy's port, such as setup pkgproxy run earlier. Then it installs the image-garden snap, in devmode, through pkgproxy, so the snap and its components are cached too. Its install hook adds QEMU for the runner and the bundled pkgproxy component.
Details and sources
Details
/dev/kvm:sudo chmod -v 666 /dev/kvmwhen it exists, otherwise a notice and no hardware acceleration (image-garden-action action.yaml:153-160). There is no udev rule.- If something already listens on port 3142, the action uses it ("external" mode) and does not save its cache; otherwise it installs the pkgproxy snap ("snap" mode) (image-garden-action action.yaml:176-188).
- snapd is refreshed or installed from
snapd-channel, pkgproxy frompkgproxy-channel(image-garden-action action.yaml:227-232). sudo snap install --devmode --channel=… image-garden; snapd downloads the snap, its base and the components its hooks install through pkgproxy (image-garden-action action.yaml:370-375).- The snap's install hook installs
+qemu-<arch>and+pkgproxy(install:6-14).
Read more
04 Pointing image-garden at pkgproxy
snap set image-garden pkgproxy=snap after snap connect image-garden:pkgproxy-sockets pkgproxy:sockets, or pkgproxy=http://HOST-IP:3142; plus GARDEN_DL_PKGPROXY=http://127.0.0.1:3142.
Next, image-garden has to know which pkgproxy to use. With the pkgproxy snap, the action connects the two snaps' sockets interface and sets the image-garden option to snap. If another pkgproxy was already listening, it sets the option to that proxy's URL instead, on the runner's own address, because inside a guest, localhost would be the guest itself. Either way, guests get pkgproxy through their cloud-init. And one variable in the job's environment sends image-garden's own image downloads through pkgproxy too, fetched by their checksum.
Details and sources
Details
- Snap mode:
snap connect image-garden:pkgproxy-sockets pkgproxy:socketsandsnap set image-garden pkgproxy=snap. External mode:snap set image-garden pkgproxy=http://<first address of hostname -I>:3142, because 127.0.0.1 would be the guest itself (image-garden-action action.yaml:376-390).HOST-IPon the slide stands for that address. GARDEN_DL_PKGPROXY=http://127.0.0.1:3142goes into the job's environment, so image-garden downloads pristine images through pkgproxy by their checksum (image-garden-action action.yaml:391-393).- In image-garden,
pkgproxy=snapand a URL each set what the guests' cloud-init and QEMU are given (010-qemu.mk:20-44);GARDEN_DL_PKGPROXYnames the pkgproxy for image downloads (image-garden:31-58).
Read more
05 Make the image, then run spread
image-garden make SYSTEM.$(uname -m).qcow2 …, then spread -v garden:SYSTEM:TASKS. Uploaded: spread.log always, artifacts, and .image-garden/*.log on failure.
Now the work. The action aliases the snap's spread as plain spread, and sets the spread variant if you asked for one. In your spread directory it makes the system's image, with the architecture from uname -m, and folds that long first boot in the log. Then it runs spread on your backend, the system and your tasks. spread's full output goes to spread dot log, which is always uploaded. Your spread artifacts are uploaded too, with colons renamed for GitHub. If the job fails, the boot logs from .image-garden come along as well.
Details and sources
Details
snap alias image-garden.spread spread, andsnap set image-garden spread-variant=…when the input is set (image-garden-action action.yaml:394-401).- In
spread-subdir, a dry run, thenimage-garden makeofSYSTEM.$(uname -m).qcow2,.run,.user-data,.meta-dataand.seed.iso, with the first boot folded in the log (image-garden-action action.yaml:412-435). Withcache-prepared-imagesit also runsimage-garden rebase SYSTEM.$(uname -m).qcow2(image-garden-action action.yaml:436-442). spread [-artifacts DIR] -v [-live] BACKEND:SYSTEM:TASKS, piped throughtee spread.log; the exit status is spread's (image-garden-action action.yaml:443-464).- Uploads:
spread.logalways; artifacts with every:renamed to_;.image-garden/*.logwhen the job failed (image-garden-action action.yaml:465-489).
Read more
06 What is kept between jobs
Two GitHub caches hold pkgproxy's cache: images per garden system, and packages per cache-scope. save-cache defaults to default-branch; cache-prepared-images is opt-in.
A GitHub runner starts empty every time, so the action keeps pkgproxy's cache in the GitHub cache. It's restored at the start and, when the job added to it, saved at the end, in two parts. The cloud images are one part, per garden system. Everything else, the snaps and the guests' packages, is the other. By default only runs on your default branch save, so pull requests reuse its cache without crowding it out. Caching your prepared images is optional, and has its own deck.
Details and sources
Details
- The pkgproxy cache is two GitHub caches: pristine images, keyed per
runner.archandgarden-system, and packages (everything else), keyed percache-scope, which defaults to the garden system (image-garden-action action.yaml:189-208; image-garden-action README.md:160-176). save-cacheisdefault-branch(default),alwaysornever(image-garden-action action.yaml:21-23); pull requests and other branches restore the default branch's cache but never save their own; failed and cancelled jobs save too; each part is saved only when the job added to it (image-garden-action README.md:188-197).cache-prepared-imagescaches.image-garden/itself, keyed on the hash of.image-garden.mk(image-garden-action action.yaml:116-123).- The archives keep the extended attributes pkgproxy relies on (
pkgproxy-cachein the action).
Read more
07 Where to go from here
Next: the action's caches, why pkgproxy, the pkgproxy setting, spread variants and GitLab versus GitHub.
So one step in your workflow installs three snaps, opens KVM, points image-garden at pkgproxy, makes the image, runs spread and uploads what you need to read a failure. For the caches in depth, with prepared images and rebase, go to the caching deck. To see why pkgproxy is the cache at all, there's its own deck. The snap decks explain the pkgproxy setting and the spread variants. And the comparison deck puts this job side by side with the same one on GitLab.
Details and sources
Roads from here
RUNTIME · make rules and the CLI
- image-garden rebase — why prepared images need rebase
BUILT ON IT · CI integrations
- How the action caches — the two pkgproxy caches and prepared images in detail
- Container on GitLab, snap on GitHub — the same job on GitLab
PKGPROXY · caching for guests
- What pkgproxy gives a garden — why pkgproxy is the cache
SNAP · snap-only features
- The pkgproxy snap setting — pkgproxy=snap and pkgproxy=URL
- Choosing a spread — spread or spread-plus
pkgproxy map ↗
- Hosted CI runners — on the pkgproxy map, pkgproxy on hosted CI runners