The OCI container

one Dockerfile, three stages for CI operators

The container image is built by the Dockerfile in image-garden's tree in three stages, QEMU from a checked tarball with the guestfwd patch, image-garden built and checked, and a clean Ubuntu that gets both, and is published for x86_64 and aarch64 hosts on every release tag.

On the map: container

▶ Watch this deck — 7 scenes, ~5 min

01 Built from the tree, in three stages

ONE DOCKERFILE a guest in your CI job which QEMU? built from what? the image-garden tree Dockerfile build-qemu build-garden FROM ubuntu:latest the registry image-garden:TAG image-garden:latest pulled snap/snapcraft.yaml the snap, from the same tree
The image-garden tree holds the Dockerfile and the snap's snapcraft.yaml. The Dockerfile has three stages, build-qemu, build-garden and FROM ubuntu:latest; the pipeline publishes the last one to the registry as image-garden:TAG and image-garden:latest, which is where the guest in your CI job comes from. The snap is built from the same tree.

The image is built by the Dockerfile in image-garden's tree, in three stages, and published with every release.

Say your CI jobs run in the image-garden container, and one day a guest behaves differently. Which QEMU was that, and where did it come from? The answer is in one file: the Dockerfile at the top of image-garden's tree. It builds the image in three stages, and the project builds it on every change and publishes it with every release. So nobody packages the container later, somewhere else. It comes from the same tree as the snap. This deck follows the Dockerfile from its first line to the registry, and ends with building the image yourself.

Details and sources

Details

  • The image is defined by the Dockerfile at the top of the tree, in three stages: build-qemu, build-garden and the final stage, each FROM ubuntu:latest (Dockerfile:5, Dockerfile:58, Dockerfile:73).
  • image-garden's pipeline builds it in every pipeline (docker-arch) and publishes it for a tag only (docker-manifest) (.gitlab-ci.yml:1231-1278).
  • The snap is built from the same tree, by snap/snapcraft.yaml, with the same QEMU patch (px-patch).
  • What the finished image contains, and how to run it, is in running the container; this deck is about how it is made.

02 Two stages build, the third one ships

THREE STAGES build-qemu Ubuntu + compilers and libraries QEMU into /usr/local build-garden Ubuntu + shellcheck and mypy make make check make install a check fails: the build stops, no image FROM ubuntu:latest a clean start what image-garden runs /usr/local QEMU and image-garden, copied from both stages left behind: compilers, shellcheck, mypy the image carries the results
Stage build-qemu builds QEMU into /usr/local with compilers and libraries. Stage build-garden runs make, make check and make install with shellcheck and mypy; a failing check means no image. The final stage starts from a clean Ubuntu, installs what image-garden runs and copies /usr/local from both; compilers, shellcheck and mypy are left behind.

build-qemu compiles QEMU, build-garden runs make, make check and make install, and the final stage copies /usr/local from both onto a clean Ubuntu.

Only the last of the three stages is the image you pull. The first, build-qemu, starts from Ubuntu with compilers and libraries, and builds QEMU into /usr/local. The second, build-garden, starts from Ubuntu again, with shellcheck and mypy. It runs make, then the project's own checks, then make install. If a check fails, there's no image. The third stage starts clean, installs only what image-garden runs, and copies /usr/local out of each of the first two. The compilers and the checkers stay behind, so the image carries the results and not the tools that made them.

Details and sources

Details

  • build-qemu installs bison, flex, g++, meson, ninja and the development libraries QEMU needs, then builds and installs QEMU with --prefix=/usr/local (Dockerfile:5-54).
  • build-garden installs curl, make, mypy, shfmt, python3, qemu-utils (for the rebase test), shellcheck and xz-utils, adds the tree, and runs make --warn-undefined-variables && make check && make install --warn-undefined-variables (Dockerfile:58-71). A failing command fails the build (derived from Docker's RUN).
  • The final stage installs ca-certificates, curl, git, git-restore-mtime, libcurl3t64-gnutls, libfdt1, libslirp0, make, ovmf, swtpm, xorriso and xz-utils, then copies /usr/local from build-qemu and from build-garden (Dockerfile:73-104).
  • Derived: in a multi-stage build only the last stage becomes the image, so the compilers and the checkers of the first two stages are not in it.
  • The checks make check runs are listed in a source build.

03 QEMU from a checked tarball

THE QEMU STAGE the QEMU release tarball sha256 checked on download guestfwd-unix-socket.patch the snap's patch too configure --without-default-features --enable-kvm --enable-tcg --enable-system --enable-slirp --enable-tpm --enable-virtfs --enable-vhost-{crypto,kernel,net,user} --enable-tools --enable-attr --enable-pie --enable-strip --target-list=... x86_64-softmmu aarch64-softmmu riscv64-softmmu s390x-softmmu ppc64-softmmu loongarch64-softmmu stripped binaries; what is not enabled is not built, such as a graphical display
The QEMU release tarball is checked against its sha256, patched with guestfwd-unix-socket.patch, and configured with --without-default-features plus KVM, TCG, system emulation, slirp, TPM, virtfs, vhost, the tools, attr, PIE and strip, for six targets: x86_64, aarch64, riscv64, s390x, ppc64 and loongarch64. The binaries are stripped; a graphical display is not built.

The tarball is pinned by its sha256, patched with guestfwd-unix-socket.patch, configured with --without-default-features plus a short list of features, for six targets, and stripped.

The QEMU stage starts with the release tarball. Docker checks its sha256 when it downloads it, so a tarball that changed stops the build. Then the guestfwd patch from the tree's qemu-patches directory goes in, the same one the snap carries. configure starts with every optional feature off and turns on only what image-garden uses: KVM and TCG, the slirp user network, TPM, virtfs, vhost, and the tools, which give you qemu-img. The target list names six system emulators, one per guest architecture, and the binaries are stripped. Whatever isn't turned on isn't there, a graphical display among them.

Details and sources

Details

  • ADD --checksum=sha256:… https://download.qemu.org/qemu-11.1.3.tar.xz /src/: Docker verifies the checksum of the download (Dockerfile:6).
  • qemu-patches/guestfwd-unix-socket.patch is copied in and applied with patch -p1 (Dockerfile:29-32); the comment says it is the same patch as the snap's QEMU.
  • configure --without-default-features with --enable-attr --enable-kvm --enable-pie --enable-slirp --enable-strip --enable-system --enable-tcg --enable-tools --enable-tpm --enable-vhost-crypto --enable-vhost-kernel --enable-vhost-net --enable-vhost-user --enable-virtfs, and QEMU_TARGET_LIST=x86_64-softmmu,aarch64-softmmu,riscv64-softmmu,s390x-softmmu,ppc64-softmmu,loongarch64-softmmu (Dockerfile:33-54).
  • The README: "the same minimal configuration as the snap package", stripped binaries, "KVM, TCG, slirp, virtfs, vhost, TPM" (README.md:351-358).
  • Derived: with --without-default-features, display back ends such as GTK, SDL and VNC are not built. image-garden's .run scripts use -nographic by default (mk/010-qemu.mk:13), so a QEMU_DISPLAY_OPTION that asks for a window does not work in the container.
  • The QEMU version is in the Dockerfile and the README (README.md:99).

04 What the image sets for you

SET FOR YOU ENV GARDEN_CACHE_DIR=/var/cache/image-garden VOLUME /var/cache/image-garden downloads live outside the container's layers ENV GARDEN_*_PREFIX=/usr/local, four of them x86_64 aarch64 riscv64 loongarch64 EDK2 firmware built with QEMU, in /usr/local apt install ... ovmf swtpm ... Secure Boot and TPM: the +tpm systems rm -f /usr/local/share/doc/image-garden/NEWS.snappy* news: NEWS and NEWS.docker defaults only: docker run -e NAME=VALUE replaces any of them
The final stage sets GARDEN_CACHE_DIR to /var/cache/image-garden and makes it a volume, so downloads live outside the container's layers; four GARDEN_*_PREFIX variables, for x86_64, aarch64, riscv64 and loongarch64, point at /usr/local, for the EDK2 firmware built with QEMU; it installs swtpm and ovmf for the +tpm systems; it deletes NEWS.snappy, so image-garden news shows NEWS and NEWS.docker. All are defaults that docker run -e replaces.

GARDEN_CACHE_DIR=/var/cache/image-garden (a volume), GARDEN_*_PREFIX=/usr/local for the EDK2 firmware, ovmf and swtpm, and no NEWS.snappy.

The last stage also makes a few choices for you. GARDEN_CACHE_DIR points at /var/cache/image-garden, and that directory is a volume, so downloads live outside the container's own layers. Four prefix variables point at /usr/local, where the QEMU stage left its EDK2 firmware, one for each architecture that boots with it; the firmware deck has the details. Ubuntu's swtpm and ovmf packages are there for the TPM systems. And the snap's release notes are deleted, so image-garden news shows the main notes and the container's own. These are environment defaults: pass your own with -e when you need to.

Details and sources

Details

  • ENV GARDEN_CACHE_DIR=/var/cache/image-garden and VOLUME /var/cache/image-garden (Dockerfile:96-97); mount a directory there to keep downloads, as running the container shows.
  • GARDEN_X86_64_PREFIX, GARDEN_AARCH64_PREFIX, GARDEN_RISCV64_PREFIX and GARDEN_LOONGARCH64_PREFIX are /usr/local (Dockerfile:99-102), one for each architecture whose firmware list reads a prefix. The x86_64 makefile, for example, appends $(GARDEN_X86_64_PREFIX)/share/qemu/edk2-x86_64-code.fd to its search list (mk/010-qemu-x86_64.mk:277-280); see firmware for the order. s390x and ppc64 take no firmware file, and the image sets no prefix for them.
  • ovmf and swtpm come from Ubuntu (Dockerfile:78-91), for the Ubuntu Core systems with a TPM (NEWS.docker:5-6).
  • rm -f /usr/local/share/doc/image-garden/NEWS.snappy* (Dockerfile:104); image-garden news prints NEWS, NEWS.snappy and NEWS.docker, skipping those that are missing (image-garden:912-942).
  • git config --system alias.whatchanged 'log --raw --no-merges': git restore-mtime runs git whatchanged, which the image's git refuses to run (Dockerfile:92-95).
  • Derived: ENV values are defaults; docker run -e NAME=VALUE replaces them for that container.

05 From a pipeline to the registry

TO THE REGISTRY docker-arch on x86_64 docker-arch on aarch64 the ci/ repository pipeline-ID-ARCH every pipeline: a merge request shows that the image builds docker-manifest on a release tag only image-garden:TAG image-garden:latest FROM ubuntu:latest the current LTS on the day of the build a later rebuild may start from a newer base: name the release tag
In every pipeline docker-arch builds the image natively on an x86_64 and an aarch64 runner and pushes both to the ci/ repository, so a merge request shows that the image builds. On a release tag only, docker-manifest joins them into image-garden:TAG and image-garden:latest. Every stage starts FROM ubuntu:latest, the current LTS on the day of the build, so a later rebuild may get a newer base: name the release tag.

docker-arch builds per architecture in every pipeline; docker-manifest joins them, for a tag only, as the tag and latest. The base is ubuntu:latest at build time.

Here's how an image reaches the registry. Every pipeline builds it twice, natively, on an x86_64 runner and on an arm sixty-four one, and pushes both to a CI repository, named after the pipeline. So a merge request already shows that the image builds. Only a release tag goes further: one more job joins the two into a single image, tagged with the release and with latest. All three stages start from Ubuntu's latest image, the current LTS on the day of the build. A release's image stays as it was published, but the same tree built later may start from a newer base. In your jobs, name the release tag.

Details and sources

Details

  • docker-arch runs on runners tagged [$ARCH, docker] for x86_64 and aarch64, builds with docker build --pull and pushes $CI_REGISTRY_IMAGE/ci:pipeline-$CI_PIPELINE_ID-$ARCH; "Every pipeline builds them, so a merge request shows that the image builds" (.gitlab-ci.yml:1231-1255).
  • docker-manifest joins the two with docker buildx imagetools create as $CI_REGISTRY_IMAGE:$CI_COMMIT_TAG and :latest, for a tag only and never in a schedule (.gitlab-ci.yml:1257-1278). The image is registry.gitlab.com/zygoon/image-garden.
  • Every stage is FROM ubuntu:latest; the README: it "resolves to the current Ubuntu LTS release" (README.md:346-349). --pull fetches the newest base at build time. The image carries LABEL version with the release (Dockerfile:74).
  • Derived: a published tag is not rebuilt; a later build of the same tree may start from a newer ubuntu:latest. latest moves with every release.

06 Building the image yourself

YOUR OWN BUILD $ docker build -t image-garden:latest . $ podman build -t image-garden:latest . $ make docker-test-smoke docker build -t image-garden:latest . docker run ... ldd .../qemu-system-* docker run ... image-garden news docker run ... image-garden make -C /tmp ubuntu-cloud-24.04.$(uname -m).run the published image, at a tag spread-gitlab-ci's Dockerfile FROM .../image-garden:TAG ADD --checksum=... spread ADD --checksum=... spread-plus ADD --checksum=... spread-gitlab-report
docker build or podman build with -t image-garden:latest in the tree builds the image; make docker-test-smoke builds it, runs ldd on the emulators, image-garden news, and image-garden make -C /tmp ubuntu-cloud-24.04.$(uname -m).run. To add tools, spread-gitlab-ci's Dockerfile starts FROM the published image at a tag and adds spread, spread-plus and spread-gitlab-report, each with a checksum.

docker build -t image-garden:latest . in the tree, or make docker-test-smoke to build and try it. To extend it, start FROM registry.gitlab.com/zygoon/image-garden:TAG, as spread-gitlab-ci does.

You can build the image yourself, from a checkout of the tree: docker build, in its top directory. Podman builds it too. The tree's Makefile has a smoke test that does exactly that. It builds the image, checks that the emulators find their libraries, prints the news, then makes an Ubuntu 24.04 cloud system for your machine's architecture in /tmp. To add tools instead, start from the published image. spread-gitlab-ci does just that: its Dockerfile begins from the image-garden image at a release tag, and adds spread, spread-plus and a report tool, each pinned by its checksum.

Details and sources

Details

  • docker-test-smoke runs docker build -t image-garden:latest ., then three docker runs with /dev/kvm when the host has it and the cache volume: ldd on qemu-img and the six qemu-system-*, image-garden news, and image-garden make -C /tmp ubuntu-cloud-24.04.$(uname -m).run; docker-test-all makes all -k (Makefile:137-166).
  • "The OCI container can be built with either Docker or Podman now." (NEWS.docker:48).
  • spread-gitlab-ci's image starts FROM registry.gitlab.com/zygoon/image-garden:v0.7.0 AS base and adds spread, spread-plus and spread-gitlab-report with ADD --checksum, per architecture (spread-gitlab-ci Dockerfile:18-34). It builds on an older image-garden than the one this map describes (see spread-gitlab-ci).

07 Where to go from here

ROADS The OCI container Running it /dev/kvm and a cache volume In CI runners of your own pkgproxy from it a URL both can reach The guestfwd patch what the QEMU stage adds Firmware how the prefixes are searched spread-gitlab-ci an image built on this one
Six roads from The OCI container: running the container, the container in CI, pkgproxy from a container, the guestfwd patch, firmware, and spread-gitlab-ci

Next: running the container, the container in CI, pkgproxy from a container, the guestfwd patch, firmware, and spread-gitlab-ci.

So the container is three stages of one Dockerfile: QEMU from a checked tarball with the guestfwd patch, image-garden built and checked, and a clean Ubuntu that gets both. Next, run it on your own project, with the KVM device and a cache volume, or in GitLab CI on runners of your own. The pkgproxy deck shows how the container and its guests reach a proxy by URL. The patch deck explains what the QEMU stage adds, the firmware deck how the prefixes are searched, and spread-gitlab-ci is an image built on top of this one.

Details and sources

Read more

Basis of the statements on this page

Read in the Dockerfile, Makefile, NEWS.docker, README and .gitlab-ci.yml of image-garden at the pinned commit ("code"). The image was not built for this map; what follows from Docker's multi-stage builds is marked "derived" in the slides' details.

The stages

StageStarts fromInstallsRunsLeaves for the image
build-qemuubuntu:latestbison, bzip2, device-tree-compiler, flex, g++, gnutls-dev, libcurl4-gnutls-dev, libfdt-dev, libglib2.0-dev, libncurses-dev, libslirp-dev, make, meson, ninja-build, patch, python3-pip, uuid-dev, xz-utils, zlib1g-devthe checked tarball, the guestfwd patch, configure, make, make installQEMU in /usr/local
build-gardenubuntu:latestcurl, make, mypy, shfmt, python3, qemu-utils, shellcheck, xz-utilsmake, make check, make installimage-garden in /usr/local
finalubuntu:latestca-certificates, curl, git, git-restore-mtime, libcurl3t64-gnutls, libfdt1, libslirp0, make, ovmf, swtpm, xorriso, xz-utilsthe git alias, rm of NEWS.snappy*the image

QEMU's configuration

configure --without-default-features \ --enable-attr --enable-kvm --enable-pie --enable-slirp \ --enable-strip --enable-system --enable-tcg --enable-tools \ --enable-tpm --enable-vhost-crypto --enable-vhost-kernel \ --enable-vhost-net --enable-vhost-user --enable-virtfs \ --prefix=/usr/local \ --target-list=x86_64-softmmu,aarch64-softmmu,riscv64-softmmu,s390x-softmmu,ppc64-softmmu,loongarch64-softmmu

Environment of the image

VariableValueRead by
GARDEN_CACHE_DIR/var/cache/image-garden (a volume)image-garden and its makefile
GARDEN_X86_64_PREFIX/usr/localmk/010-qemu-x86_64.mk
GARDEN_AARCH64_PREFIX/usr/localmk/010-qemu-aarch64.mk
GARDEN_RISCV64_PREFIX/usr/localmk/010-qemu-riscv64.mk
GARDEN_LOONGARCH64_PREFIX/usr/localmk/010-qemu-loongarch64.mk

No ENTRYPOINT or CMD is set.

Building it

# in a checkout of the image-garden tree docker build -t image-garden:latest . # or: podman build ... make docker-test-smoke # build, then three test runs

Publishing

JobWhenResult
docker-archevery pipeline, on an x86_64 and an aarch64 runnerci:pipeline-ID-ARCH in the project's registry
docker-manifesta tag, never a scheduleregistry.gitlab.com/zygoon/image-garden:TAG and :latest

Roads from here

ARCHITECTURES · machines and firmware

BUILT ON IT · CI integrations

PKGPROXY · caching for guests

CONTAINER · OCI image