image-garden-action for GitHub

image-garden-action for CI operators

image-garden-action runs spread tests on GitHub runners. It installs snapd, the pkgproxy snap and the image-garden snap, opens /dev/kvm, points image-garden at pkgproxy, makes the garden system's image and runs spread on it, then uploads spread.log, artifacts and boot logs, and keeps the pkgproxy cache in the GitHub cache.

On the map: built on it

▶ Watch this deck — 7 scenes, ~5 min

Worth seeing first

Part of journeys

01 Your spread tests on GitHub

ON GITHUB spread.yaml · backend garden passes on your machine by hand, on every runner: install snap pkgproxy make image spread keep downloads one step: zyga/image-garden-action garden-system: debian-cloud-13 the same spread command you run at home, on a fresh runner
Done by hand on every runner: install the snap, start pkgproxy, make the image, run spread, keep the downloads; image-garden-action does all of it in one workflow step for the garden system you name, running the same spread command as at home

zyga/image-garden-action installs image-garden on a GitHub runner and runs your spread tests on one garden system per job.

You've got a spread.yaml with the garden backend, and the tests pass on your machine. On GitHub you'd have to repeat what you did by hand on every runner: install the snap, get pkgproxy going, make the image, run spread, and keep the downloads for next time. The image-garden action does all of that in one step of your workflow. You tell it which garden system to use, and it runs the same spread command you run at home, on a fresh runner, every time. Let's follow one job from checkout to the logs.

Details and sources

Details

02 The workflow

THE WORKFLOW jobs: debian-13: runs-on: ubuntu-latest steps: - uses: actions/checkout@v6 with: fetch-depth: 0 the git history gives .image-garden.mk its real change time; without it, a warning - uses: zyga/image-garden-action@v0 with: garden-system: debian-cloud-13 spread-variant: plus spread-tasks: tests/… optional
A workflow with one job, debian-13, on ubuntu-latest: actions/checkout with fetch-depth 0, so the action can restore the time .image-garden.mk last changed, then zyga/image-garden-action@v0 with garden-system debian-cloud-13; optional inputs spread-variant, spread-tasks and the cache inputs

One job per system: actions/checkout with fetch-depth: 0, then zyga/image-garden-action@v0 with garden-system. The other inputs are optional.

Here's a whole workflow. One job per system: this one is Debian 13. The first step checks out your code, and note fetch-depth: 0. The action uses the git history to give your .image-garden.mk its real change time, so make can tell that a cached image is still current. Without the history it warns. The second step uses the action, and the one input it needs is garden system. Others are optional: spread variant picks spread plus, spread tasks runs a subset, and the cache inputs come later.

Details and sources

Details

  • The README's example workflow: one job per system, actions/checkout with fetch-depth: 0 ("essential for git restore-mtime to work correctly"), then zyga/image-garden-action@v0 with garden-system (image-garden-action README.md:405-432).
  • The inputs, with defaults (image-garden-action action.yaml:8-56): garden-system is required; spread-variant, spread-live, spread-subdir, spread-backend (garden), spread-tasks, spread-artifacts; the cache inputs cache-prepared-images, save-cache, save-package-cache, cache-scope, cache-key-prefix; and the three snap channels.
  • When the project has a .image-garden.mk, the action runs git restore-mtime on it, so make sees when it last changed in git, not when it was checked out; without the history it warns "check out with fetch-depth: 0" (image-garden-action action.yaml:124-152).

03 What it installs on the runner

ON THE RUNNER sudo chmod 666 /dev/kvm QEMU gets hardware acceleration no /dev/kvm: emulated, much slower snapd the pkgproxy snap unless port 3142 is taken, as by setup-pkgproxy snap install --devmode image-garden through pkgproxy: cached + QEMU for this runner + the pkgproxy component its install hook
On the runner the action opens /dev/kvm to every user, or warns and runs emulated without it; installs or refreshes snapd; installs the pkgproxy snap unless port 3142 is already taken; installs the image-garden snap in devmode through pkgproxy, whose install hook adds QEMU and the pkgproxy component

The action makes /dev/kvm usable, installs snapd and the pkgproxy snap (unless port 3142 is taken), then snap install --devmode image-garden.

The runner is an ordinary Ubuntu machine, so the action prepares it. If /dev/kvm exists, it opens it to every user, so QEMU gets hardware acceleration; without it, everything runs emulated and much slower. It installs or refreshes snapd, then the pkgproxy snap, unless something already listens on pkgproxy's port, such as setup pkgproxy run earlier. Then it installs the image-garden snap, in devmode, through pkgproxy, so the snap and its components are cached too. Its install hook adds QEMU for the runner and the bundled pkgproxy component.

Details and sources

Details

04 Pointing image-garden at pkgproxy

POINTING AT PKGPROXY the pkgproxy snap snap connect image-garden:pkgproxy-sockets pkgproxy:sockets snap set image-garden pkgproxy=snap another pkgproxy already on port 3142 snap set image-garden pkgproxy=http://HOST-IP:3142 (hostname -I) not 127.0.0.1: inside a guest, that is the guest itself guests: pkgproxy through their cloud-init GARDEN_DL_PKGPROXY=http://127.0.0.1:3142 image-garden's own image downloads, by checksum
With the pkgproxy snap the action connects image-garden's pkgproxy-sockets plug and sets pkgproxy=snap; with another pkgproxy on port 3142 it sets pkgproxy to the runner's own address, since localhost in a guest is the guest; guests get pkgproxy through cloud-init; GARDEN_DL_PKGPROXY sends image downloads through pkgproxy too

snap set image-garden pkgproxy=snap after snap connect image-garden:pkgproxy-sockets pkgproxy:sockets, or pkgproxy=http://HOST-IP:3142; plus GARDEN_DL_PKGPROXY=http://127.0.0.1:3142.

Next, image-garden has to know which pkgproxy to use. With the pkgproxy snap, the action connects the two snaps' sockets interface and sets the image-garden option to snap. If another pkgproxy was already listening, it sets the option to that proxy's URL instead, on the runner's own address, because inside a guest, localhost would be the guest itself. Either way, guests get pkgproxy through their cloud-init. And one variable in the job's environment sends image-garden's own image downloads through pkgproxy too, fetched by their checksum.

Details and sources

Details

  • Snap mode: snap connect image-garden:pkgproxy-sockets pkgproxy:sockets and snap set image-garden pkgproxy=snap. External mode: snap set image-garden pkgproxy=http://<first address of hostname -I>:3142, because 127.0.0.1 would be the guest itself (image-garden-action action.yaml:376-390). HOST-IP on the slide stands for that address.
  • GARDEN_DL_PKGPROXY=http://127.0.0.1:3142 goes into the job's environment, so image-garden downloads pristine images through pkgproxy by their checksum (image-garden-action action.yaml:391-393).
  • In image-garden, pkgproxy=snap and a URL each set what the guests' cloud-init and QEMU are given (010-qemu.mk:20-44); GARDEN_DL_PKGPROXY names the pkgproxy for image downloads (image-garden:31-58).

05 Make the image, then run spread

MAKE, THEN SPREAD snap alias image-garden.spread spread · spread-variant=… image-garden make debian-cloud-13.$(uname -m).qcow2 … .run … the first boot's long console, folded in the job log spread -v garden:debian-cloud-13:TASKS | tee spread.log spread.log always spread artifacts colons renamed .image-garden/*.log when the job fails
The action aliases image-garden.spread as spread and sets the spread variant; it makes SYSTEM.$(uname -m).qcow2 and its other files in the spread directory, folding the first boot in the log; then runs spread -v garden:SYSTEM:TASKS; spread.log is always uploaded, artifacts with colons renamed, and .image-garden/*.log on failure

image-garden make SYSTEM.$(uname -m).qcow2 …, then spread -v garden:SYSTEM:TASKS. Uploaded: spread.log always, artifacts, and .image-garden/*.log on failure.

Now the work. The action aliases the snap's spread as plain spread, and sets the spread variant if you asked for one. In your spread directory it makes the system's image, with the architecture from uname -m, and folds that long first boot in the log. Then it runs spread on your backend, the system and your tasks. spread's full output goes to spread dot log, which is always uploaded. Your spread artifacts are uploaded too, with colons renamed for GitHub. If the job fails, the boot logs from .image-garden come along as well.

Details and sources

Details

06 What is kept between jobs

BETWEEN JOBS an empty runner the GitHub cache restore at the start save at the end, if it grew images · per system packages · per cache-scope save-cache: default-branch pull requests restore the default branch's cache, and never save their own cache-prepared-images: true opt-in, its own deck
Each job restores pkgproxy's cache from the GitHub cache and saves it at the end when it grew, in two parts: images per garden system and packages per cache-scope; by default only the default branch saves, pull requests restore; caching prepared images is opt-in

Two GitHub caches hold pkgproxy's cache: images per garden system, and packages per cache-scope. save-cache defaults to default-branch; cache-prepared-images is opt-in.

A GitHub runner starts empty every time, so the action keeps pkgproxy's cache in the GitHub cache. It's restored at the start and, when the job added to it, saved at the end, in two parts. The cloud images are one part, per garden system. Everything else, the snaps and the guests' packages, is the other. By default only runs on your default branch save, so pull requests reuse its cache without crowding it out. Caching your prepared images is optional, and has its own deck.

Details and sources

Details

07 Where to go from here

ROADS GitHub action Caching on GitHub prepared images, rebase Why pkgproxy download once pkgproxy= snap, URL, none spread-variant spread or spread-plus GitLab vs GitHub container vs snap
Five roads from image-garden-action: the action's caches, why pkgproxy, the pkgproxy setting, spread variants, and GitLab versus GitHub

Next: the action's caches, why pkgproxy, the pkgproxy setting, spread variants and GitLab versus GitHub.

So one step in your workflow installs three snaps, opens KVM, points image-garden at pkgproxy, makes the image, runs spread and uploads what you need to read a failure. For the caches in depth, with prepared images and rebase, go to the caching deck. To see why pkgproxy is the cache at all, there's its own deck. The snap decks explain the pkgproxy setting and the spread variants. And the comparison deck puts this job side by side with the same one on GitLab.

Details and sources

Roads from here

RUNTIME · make rules and the CLI

BUILT ON IT · CI integrations

PKGPROXY · caching for guests

SNAP · snap-only features

pkgproxy map ↗