Projects built on image-garden

four components, one action for everyone

Five projects put image-garden into CI. Four GitLab CI components build on the image-garden container to pack snaps, rocks and charms in a throwaway VM or to run spread; image-garden-action installs the snap on GitHub runners and runs spread. All of them rely on the same few commands and on pkgproxy as their cache.

On the map: built on it

▶ Watch this deck — 5 scenes, ~4 min

Worth seeing first

01 The same machines in CI

BUILT ON IT your laptop Debian Fedora spread tests pass every merge request that machine needs: image-garden and QEMU KVM a place to keep downloads five small projects do that wiring four GitLab CI components · the container one GitHub action · the snap
spread tests that pass on a laptop on Debian and Fedora garden VMs should also run on every merge request; a CI machine needs image-garden, QEMU, KVM and a place for downloads; four GitLab CI components on the container and one GitHub action on the snap provide that

Five projects run image-garden in CI: four GitLab CI components on the container, one GitHub action on the snap.

Say your spread tests pass on your laptop, on a garden VM of Debian and one of Fedora. Sooner or later you want them to run on every merge request too, on a machine you don't sit at. That machine needs image-garden, QEMU, a way to reach KVM, and somewhere to keep downloads. Five small projects do that wiring for you, so a CI job is a few lines of configuration. Four of them are GitLab CI components built on the image-garden container. One is a GitHub action built on the snap. This deck is the map of the five.

Details and sources

Details

  • image-garden needs /dev/kvm for full speed; without it QEMU emulates the CPU, many times slower. GitLab.com's hosted runners do not give jobs /dev/kvm, so image-garden jobs, and every job built on it, run on runners of your own that pass the device in (README.md:778-787).
  • The five projects and the commits this map read: snapcraft-gitlab-ci 91b5e29, rockcraft-gitlab-ci e8fd725, charmcraft-gitlab-ci 6fc8543, spread-gitlab-ci 3d522d4, image-garden-action 2891c7a (integrations.commits in this map's repository).
  • Debian and Fedora are an example of two systems, not something the projects require.

02 Four components on the container

GITLAB: THE CONTAINER FROM registry.gitlab.com/zygoon/image-garden snapcraft-gitlab-ci · rockcraft-gitlab-ci · charmcraft-gitlab-ci copy in pack copy out in a throwaway Ubuntu VM that matches the target .snap .rock .charm spread-gitlab-ci + spread, spread-plus + a reporter your spread.yaml results in GitLab all four: runners that give the job /dev/kvm
The image-garden container is the base of four GitLab CI component images; snapcraft-, rockcraft- and charmcraft-gitlab-ci start a throwaway Ubuntu VM, copy the project in, pack a snap, rock or charm and copy it out; spread-gitlab-ci adds spread and a reporter and runs your spread.yaml; all need runners with KVM

snapcraft-, rockcraft- and charmcraft-gitlab-ci pack in a throwaway VM; spread-gitlab-ci runs your spread.yaml. Each image is built FROM the image-garden container.

On GitLab, a job runs inside a container image, so these four start from image-garden's own container and add a little. Three of them build packages. snapcraft, rockcraft and charmcraft GitLab CI each start a throwaway Ubuntu VM that matches what they build for, copy your project in, pack a snap, a rock or a charm inside it, and copy the result back out. The fourth, spread GitLab CI, adds spread and a reporter, then runs your own spread.yaml and shows the results the way GitLab can. All four need runners that give the job KVM.

Details and sources

Details

  • Each image starts from image-garden's container: FROM registry.gitlab.com/zygoon/image-garden:v0.7.2 in snapcraft-gitlab-ci (snapcraft-gitlab-ci Dockerfile:9), rockcraft-gitlab-ci (rockcraft-gitlab-ci Dockerfile:9) and charmcraft-gitlab-ci (charmcraft-gitlab-ci Dockerfile:15); spread-gitlab-ci still starts from v0.7.0 (spread-gitlab-ci Dockerfile:20).
  • Known discrepancy: spread-gitlab-ci builds on an older image-garden. None of the four uses v0.8.0 yet.
  • The craft images add the ssh client, sshpass and their *craft-vm script; charmcraft-gitlab-ci also bundles charmcraft and skopeo for publishing. spread-gitlab-ci adds checksum-pinned spread, spread-plus and spread-gitlab-report, which turns spread's output into JUnit and annotations for GitLab (spread-gitlab-ci Dockerfile:22-51).
  • The craft scripts start the VM with image-garden allocate --spread, copy the project in with tar over ssh, run pack --destructive-mode inside, and copy the result back (snapcraft-gitlab-ci bin/snapcraft-vm).

03 One action on the snap

GITHUB: THE SNAP the GitHub runner: a whole virtual machine snapd pkgproxy image-garden connected: the image-garden snap uses that pkgproxy spread -v garden:debian-cloud-13: the spread the snap carries uses: zyga/ image-garden-action garden-system: … nothing to build: the snap store and the action's version pick what you get
A GitHub job runs on a whole virtual machine; image-garden-action installs snapd, the pkgproxy snap and the image-garden snap on it, connects them, and runs the snap's spread on the one garden system the workflow names; nothing to build, the snap store and the action version decide what you get

image-garden-action installs snapd, pkgproxy and the image-garden snap on the GitHub runner, then runs the snap's spread on one garden system.

GitHub works the other way round. A job runs on a whole virtual machine, not in a container you choose, so the GitHub action installs things on that runner instead. It installs snapd, the pkgproxy snap and the image-garden snap, connects them, and then runs the spread that the snap carries. Your workflow names one garden system per job, and the action makes its image and runs your tests on it. Nothing to build, and no image of your own to keep up to date: the snap store and the action's version pick what you get.

Details and sources

Details

04 What all five rely on

WHAT THEY SHARE $ image-garden allocate --spread … <ADDRESS localhost:PORT> $ ./SYSTEM.run --query user and password for the craft tools; spread reads its own spread.yaml $ image-garden discard localhost:PORT the machine is gone no prepared images kept by default: every job starts from nothing pkgproxy cloud image · packages · snaps, between jobs the second run is fast
All five ask image-garden allocate for a machine and read its ADDRESS line, get the user and password from the .run script with --query or from spread's configuration, stop the machine with discard, and keep no prepared images by default; pkgproxy keeps the cloud image, packages and snaps between jobs

All five use allocate, its <ADDRESS> line, --query or spread's credentials, and discard. pkgproxy is their only cache between jobs.

Under the differences, all five lean on the same few things. Each one asks image-garden allocate for a machine and reads back the address line. The craft components ask the .run script for the user and password with --query, and spread reads them from its own configuration. When they are done, discard stops the machine. And none of them keeps a cache of prepared images by default. Every job starts from nothing, and pkgproxy is what makes the second run fast: it keeps the cloud image, the packages and the snaps between jobs.

Details and sources

Details

  • image-garden allocate --spread prints <ADDRESS localhost:PORT> once SSH answers (image-garden:1203).
  • SYSTEM.run --query prints the system's users and passwords (010-qemu.mk:63-92); the craft scripts read system.users.0.username and .password from it.
  • "The component keeps no cache of its own between jobs: pkgproxy is what makes the second build fast" (snapcraft-gitlab-ci README.md:205-210). The action uses pkgproxy as its only cache for downloads (image-garden-action README.md:152-158); caching prepared images is an opt-in input there.
  • The full list of what the projects expect from image-garden is in the contract.

05 Where to go from here

ROADS Built on it spread-gitlab-ci spread on GitLab GitHub action spread on GitHub *craft-gitlab-ci packing in a VM The contract what they rely on Why pkgproxy download once KVM in CI runners with /dev/kvm
Six roads from Built on it: spread-gitlab-ci, image-garden-action, the craft components, the contract, why pkgproxy, and KVM in CI

Next: spread-gitlab-ci, image-garden-action, the craft components, the contract, pkgproxy and KVM runners.

That's the family: four components on the container for GitLab, one action on the snap for GitHub, and the same allocate, query and discard underneath. If you run spread, go to spread GitLab CI or to the GitHub action next. If you pack snaps, rocks or charms, the craft deck shows the throwaway VM in detail. The contract deck lists exactly what these projects expect from image-garden. And whichever you pick, read why pkgproxy matters in CI, and how to get runners with KVM.

Details and sources

Roads from here

BUILT ON IT · CI integrations

PKGPROXY · caching for guests

CONTAINER · OCI image