UEFI firmware per system
code and vars per system for contributors
Each efi system boots from its own qcow2 copies of the host's EDK2 code and variables, found through per-architecture search lists that end at the snap's component and GARDEN_ARCH_PREFIX; ten older aarch64 systems use Fedora's EDK2, Ubuntu 14.04 and 16.04 a 2022 build, and the +tpm systems an OVMF Secure Boot pair.
On the map: architectures
▶ Watch this deck — 7 scenes, ~5 min
Worth seeing first
- Boot types — which systems are efi and what efi adds to the rules
01 Firmware of its own
efi-code.x86_64.img and efi-vars.x86_64.img are copied once from the host; each efi system gets X.efi-code.qcow2, attached read-only, and X.efi-vars.qcow2, writable.
A UEFI guest doesn't only run its firmware; it writes to it. Boot entries and settings land in a store of UEFI variables, and two systems sharing one store would trip over each other's entries. So image-garden works in two steps. Once per architecture, it copies the host's EDK2 firmware, the code and a blank variable store, next to your systems. Then every efi system, such as ubuntu-cloud-24.04.x86_64, gets its own qcow2 copy of both, named after it. The .run script attaches the code read-only and the variables writable, so whatever the guest saves stays with that one system.
Details and sources
Details
efi-code.x86_64.imgandefi-vars.x86_64.imgare copied withcp --preserve=timestampsfrom the first file found (mk/010-qemu-x86_64.mk:316-322);%.x86_64.efi-code.qcow2and%.x86_64.efi-vars.qcow2convert them with$(QEMU_IMG) convert -f raw -O qcow2, theqemu-imgthatQEMU_IMGnames (mk/010-qemu-x86_64.mk:324-329). Every UEFI architecture's file has the same rules.- The
.runscript's default firmware option: unit 0readonly=on, unit 1 writable (mk/010-qemu-x86_64.mk:67). - The README's file tables:
SYSTEM.ARCH.efi-code.qcow2is a copy ofefi-code.ARCH.img,SYSTEM.ARCH.efi-vars.qcow2holds the current variables; ten shared firmware files (README.md:827-828, README.md:839-852). - The shared and per-system files are made in the directory make runs in,
.image-garden/or the project directory (see the cache deck). - The man page names them the same way:
$SYSTEM.$ARCH.efi-code.qcow2, a copy ofefi-code.$ARCH.img, and$SYSTEM.$ARCH.efi-vars.qcow2, made fromefi-vars.$ARCH.img(image-garden.1:159-167).
Read more
02 Where the firmware comes from
QEMU_EDK2_CODE_ARCH and QEMU_EDK2_VARS_ARCH list distribution paths, then the snap component's file, then $GARDEN_ARCH_PREFIX/share/qemu. The first file that exists is copied; riscv64 copies are truncated to 32 MiB.
Where does the host's firmware come from? Each architecture's makefile lists the places where distributions install EDK2, and make takes the first file that exists. For x86_64 that's OVMF: Debian and Ubuntu's paths first, then Arch Linux's, then openSUSE's. In the snap, the list goes on to the firmware in the snap's own QEMU component, and with a prefix variable set, it ends at the firmware next to a QEMU you built yourself. The code and the variable store are each looked up in a list of their own. riscv64 adds one twist: both copies are stretched to the thirty-two megabytes its flash expects.
Details and sources
Details
- x86_64:
QEMU_EDK2_CODE_X86_64andQEMU_EDK2_VARS_X86_64, "default location on most Debian/Ubuntu systems, Arch and openSUSE", then the snap'sqemu-x86-64/share/qemu/edk2-x86_64-code.fdandedk2-i386-vars.fd, then$GARDEN_X86_64_PREFIX/share/qemu/…(mk/010-qemu-x86_64.mk:260-280). Which distribution uses which path is derived from that comment. $(firstword $(wildcard LIST))picks the first existing file for the code and, separately, for the variables (mk/010-qemu-x86_64.mk:318-322).- aarch64:
/usr/share/AAVMF/AAVMF_CODE.fd,/usr/share/qemu/aavmf-aarch64-code.bin, then the snap'sedk2-aarch64-code.fdandedk2-arm-vars.fd, then the prefix (mk/010-qemu-aarch64.mk:76-91). - riscv64:
/usr/share/qemu-efi-riscv64/RISCV_VIRT_CODE.fd,/usr/share/edk2/riscv/RISCV_VIRT_CODE.fd,/usr/share/qemu/ovmf-riscv64-code.bin, the snap's, the prefix's; both copies gettruncate --size 32MiB(mk/010-qemu-riscv64.mk:75-110). - loongarch64: see loongarch64 and OpenCloudOS.
- Every list is set with
?=, so a project can replace it in.image-garden.mkor on the make command line (derived from GNU make).
Read more
- The full lists are in this deck's reference.
03 Older Arm systems need older firmware
Ten aarch64 systems, listed in GARDEN_AARCH64_FEDORA_EDK2_SYSTEMS, use Fedora's EDK2; ubuntu-cloud-14.04 and 16.04 use Ubuntu's EDK2 from 22.04. Both come from the snap's qemu-aarch64 component or a checksummed download.
On Arm, one firmware doesn't boot everything. EDK2 from 2023 on marks the memory that boot loaders load into as not executable, and some older shims and grubs trip over that: they stop with Synchronous Exception before the kernel starts. So ten aarch64 systems, among them AlmaLinux 8, Rocky 8 and Ubuntu Core 22, boot with Fedora's EDK2, which is built to tolerate them. Ubuntu 14.04 and 16.04 need something older still: Ubuntu's EDK2 from 2022. The snap carries both builds in its aarch64 component. Elsewhere, make downloads them by checksum from image-garden's own GitLab packages.
Details and sources
Details
- The comment above
EDK2_AARCH64_FEDORA: from 2023 EDK2 marks memory that boot loaders load data into as not executable and offers the EFI memory attribute protocol, which shim 15.6 and 15.7 misuse; with Debian's and Ubuntu's EDK2 those systems stop with "Synchronous Exception" (mk/010-qemu-aarch64.mk:114-127). GARDEN_AARCH64_FEDORA_EDK2_SYSTEMS ?=almalinux-cloud-8, amazonlinux-cloud-2, amazonlinux-cloud-2023, fedora-cloud-38, opensuse-cloud-15.4, opensuse-cloud-15.5, oracle-cloud-8, rocky-cloud-8, ubuntu-core-22, ubuntu-core-22+dangerous; their.aarch64.efi-code.qcow2and.efi-vars.qcow2are copies ofQEMU_EFI-pflash.qcow2andvars-template-pflash.qcow2(mk/010-qemu-aarch64.mk:134-167).GARDEN_AARCH64_JAMMY_EDK2_SYSTEMS ?=ubuntu-cloud-14.04, ubuntu-cloud-16.04: their 4.4 kernel gives up when the memory map changes while it exits boot services ("Failed to update FDT and exit boot services" under Fedora's EDK2); Ubuntu's EDK2 2022.02 boots them (mk/010-qemu-aarch64.mk:169-213).- In the snap both builds are in the qemu-aarch64 component (
share/qemu/edk2-fedora,share/qemu/edk2-jammy); elsewheredownload-with-sha256sumfetches them into$(GARDEN_DL_DIR)/edk2-aarch64-fedora/VERSIONandedk2-aarch64-jammy/VERSIONfrom the project's GitLab generic packages (mk/010-qemu-aarch64.mk:128-161, :182-207). The pinned builds areEDK2_AARCH64_FEDORA = 20260812-10.fc45andEDK2_AARCH64_JAMMY = 2022.02-3ubuntu0.22.04.7. - Because the lists use
?=, a project can add a system to either list (derived).
Read more
04 The Secure Boot pair
OVMF_CODE_4M.secboot.fd with OVMF_VARS_4M.ms.fd, as a pair, become efi-secboot-code.x86_64.img and efi-secboot-vars.x86_64.img for the +tpm systems only.
The four Ubuntu Core +tpm systems need firmware that plain efi systems don't: Secure Boot. That's OVMF's Secure Boot build, which needs System Management Mode, and a variable store with Microsoft's keys already enrolled, so the signed shim is allowed to boot. Code and variables have to come from the same build, so here the two lists are walked as pairs, and the first pair where both files exist wins. Ubuntu's ovmf package has them, and so does the snap's x86_64 component. If neither is there, make stops and asks for edk2 firmware with Secure Boot and Microsoft keys.
Details and sources
Details
QEMU_EDK2_SECBOOT_CODE_X86_64 ?= /usr/share/OVMF/OVMF_CODE_4M.secboot.fdandQEMU_EDK2_SECBOOT_VARS_X86_64 ?= /usr/share/OVMF/OVMF_VARS_4M.ms.fd, plus the snap component'sshare/ovmfcopies;QEMU_EDK2_SECBOOT_PAIR_X86_64joins the Nth entries and takes the first pair where both exist (mk/010-qemu-x86_64.mk:282-296).efi-secboot-code.x86_64.imgandefi-secboot-vars.x86_64.imgcopy the pair or print "please install x86_64 edk2 firmware with Secure Boot and Microsoft keys (the ovmf package) - not in … with …" and fail; the check runs only when such a system is made (mk/010-qemu-x86_64.mk:298-308).- The +tpm systems'
.efi-code.qcow2and.efi-vars.qcow2are converted from them, and their.runand.qcow2useQEMU_SYSTEM_X86_64_MACHINE = q35,smm=on(mk/050-ubuntu-core.mk:55-61); "Secure Boot firmware keeps its variables in flash that only SMM may write" (mk/010-qemu-x86_64.mk:134-135). - The snap's
ovmf-x86-64part stages the pair from Ubuntu'sovmf-genericpackage into the qemu-x86-64 component (snap/snapcraft.yaml:645-665).
Read more
05 Firmware next to your own QEMU
GARDEN_X86_64_PREFIX, _AARCH64_, _RISCV64_ and _LOONGARCH64_PREFIX append PREFIX/share/qemu/edk2-* to the search lists. The container sets them to /usr/local.
If you build QEMU yourself, it installs EDK2 builds of its own, under its prefix. Four variables point image-garden there, one per UEFI architecture, such as GARDEN_X86_64_PREFIX. Each adds that prefix's code and variables to the end of its list, so a distribution's firmware still wins when it's installed. The container does exactly this: it builds QEMU into its own prefix and sets all four, and only those four. s390x and ppc64 need no prefix, as their machines take no firmware file. And because the container has Ubuntu's ovmf package, x86_64 systems there boot Ubuntu's OVMF.
Details and sources
Details
- Each UEFI architecture appends
$(GARDEN_ARCH_PREFIX)/share/qemu/edk2-…-code.fdand the matching vars file when the variable is set (mk/010-qemu-x86_64.mk:277-280, mk/010-qemu-aarch64.mk:88-91, mk/010-qemu-riscv64.mk:89-92, mk/010-qemu-loongarch64.mk:91-94). The Secure Boot lists have no prefix entry. - The container copies its QEMU build to
/usr/localand sets the four prefixes (Dockerfile:98-102); the README names the same four (README.md:360-363). - The container installs
ovmfandswtpm(Dockerfile:78-91). That Ubuntu's ovmf package has/usr/share/OVMF/OVMF_CODE_4M.fd, which comes before the prefix in the x86_64 list, is derived from the package's contents, not run. - The s390x and ppc64 makefiles have no firmware lists, so no prefix variable exists for them (sys-boot shows their command lines).
Read more
06 When the firmware is missing
A missing firmware gives please install x86_64 edk2 firmware - not in … on every make. In the snap a missing component gives install qemu-x86-64 component to use x86_64 emulation.
What if none of the files exists? Then every make prints a warning, please install x86_64 edk2 firmware, followed by the list of places it looked. You see it even when you build something else, a bios system or a list of systems, because make checks the list while it reads its rules. The build of an efi system then fails at the copy, as there's nothing to copy. In the snap, the copy first checks that the architecture's QEMU component is installed, and without it make stops and names the component to install. Either way, the fix is the same: install the firmware.
Details and sources
Details
- The prerequisite of
efi-code.x86_64.imgis$(or $(firstword $(wildcard …)),$(warning please install x86_64 edk2 firmware - not in …))(mk/010-qemu-x86_64.mk:318-322). GNU make expands prerequisites when it reads the rule, so the warning appears on every run that readsimage-garden.mk, once for each architecture whose list finds nothing (derived from GNU make). - With no prerequisite, the recipe
cp --preserve=timestamps $< $@gets one argument; thecpmessage on the slide is GNU coreutils' (derived, not run). - In the snap,
efi-code.ARCH.imghascheck-qemu-ARCH-componentas an order-only prerequisite, which runsENSURE_QEMU_SNAP_COMPONENT: "install qemu-x86-64 component to use x86_64 emulation" (mk/010-qemu-x86_64.mk:310-316, mk/010-qemu.mk:50-55). - The README lists the host tools for a source install but not the EDK2 packages by name (README.md:122-126).
Read more
07 Where to go from here
Next: boot types, Secure Boot and the TPM, the six machines, the target chain, the container in CI and the snap.
So each efi system boots from its own copy of firmware the host already has, found in a list, with older builds for older Arm systems and a Secure Boot pair for the TPM systems. From here, the boot deck shows which systems are efi at all. The Secure Boot deck shows what the pair is for. The architectures deck has the machines that load the firmware, and the target chain shows where the copies fit among the files make creates. The container and the snap each bring firmware of their own.
Details and sources
Read more
Basis of the statements on this page
Read in mk/010-qemu-x86_64.mk, mk/010-qemu-aarch64.mk, mk/010-qemu-riscv64.mk, mk/010-qemu-loongarch64.mk, mk/050-ubuntu-core.mk, snap/snapcraft.yaml, the Dockerfile and the man page of image-garden at the tag in garden.commit ("code"). Which distribution installs which path, and what make and cp print when nothing is found, are derived. Nothing was run for this page.
Search lists
The first existing file of each list is copied. Lists are tried in this order: the paths below, then the snap's component (in the snap), then $GARDEN_ARCH_PREFIX/share/qemu/ (when set).
| Arch | Code | Variables | Snap component file | Prefix file |
|---|---|---|---|---|
| x86_64 | /usr/share/OVMF/OVMF_CODE.fd, /usr/share/OVMF/OVMF_CODE_4M.fd, /usr/share/ovmf/x64/OVMF_CODE.4m.fd, /usr/share/qemu/ovmf-x86_64-4m-code.bin | the matching VARS paths | qemu-x86-64/share/qemu/edk2-x86_64-code.fd, edk2-i386-vars.fd | same names |
| aarch64 | /usr/share/AAVMF/AAVMF_CODE.fd, /usr/share/qemu/aavmf-aarch64-code.bin | the matching VARS paths | qemu-aarch64/share/qemu/edk2-aarch64-code.fd, edk2-arm-vars.fd | same names |
| riscv64 | /usr/share/qemu-efi-riscv64/RISCV_VIRT_CODE.fd, /usr/share/edk2/riscv/RISCV_VIRT_CODE.fd, /usr/share/qemu/ovmf-riscv64-code.bin | the matching VARS paths | qemu-riscv64/share/qemu/edk2-riscv-code.fd, edk2-riscv-vars.fd | same names |
| loongarch64 | /usr/share/qemu-efi-loongarch64/QEMU_EFI.fd, /usr/share/edk2/loongarch64/QEMU_EFI.fd | QEMU_VARS.fd in the same directories | qemu-loongarch64/share/qemu/edk2-loongarch64-code.fd, -vars.fd | same names |
| x86_64 Secure Boot | /usr/share/OVMF/OVMF_CODE_4M.secboot.fd | /usr/share/OVMF/OVMF_VARS_4M.ms.fd | qemu-x86-64/share/ovmf/ with the same names | none |
Files
| File | Made from | Used by |
|---|---|---|
efi-code.ARCH.img, efi-vars.ARCH.img | the first file of each list | every efi system of that architecture |
efi-secboot-code.x86_64.img, efi-secboot-vars.x86_64.img | the first complete Secure Boot pair | the four +tpm systems |
X.efi-code.qcow2 | $(QEMU_IMG) convert of the code | pflash unit 0, read-only |
X.efi-vars.qcow2 | $(QEMU_IMG) convert of the variables | pflash unit 1, writable |
$(GARDEN_DL_DIR)/edk2-aarch64-fedora/VERSION/ | download, SHA-256 checked | ten aarch64 systems (outside the snap) |
$(GARDEN_DL_DIR)/edk2-aarch64-jammy/VERSION/ | download, SHA-256 checked | ubuntu-cloud-14.04 and 16.04 on aarch64 (outside the snap) |
Knobs
| Variable | Kind | Effect |
|---|---|---|
QEMU_EDK2_CODE_ARCH, QEMU_EDK2_VARS_ARCH | make, ?= | replace a search list |
QEMU_EDK2_SECBOOT_CODE_X86_64, QEMU_EDK2_SECBOOT_VARS_X86_64 | make, ?= | replace the Secure Boot lists |
GARDEN_AARCH64_FEDORA_EDK2_SYSTEMS, GARDEN_AARCH64_JAMMY_EDK2_SYSTEMS | make, ?= | which aarch64 systems use the older builds |
GARDEN_X86_64_PREFIX, GARDEN_AARCH64_PREFIX, GARDEN_RISCV64_PREFIX, GARDEN_LOONGARCH64_PREFIX | environment | append a self-built QEMU's EDK2 to the lists |
QEMU_BOOT_FIRMWARE_OPTION | environment, at run time | replaces both pflash drives of a .run script |
QEMU_IMG | make | the qemu-img that converts the firmware copies to qcow2 |
Roads from here
UBUNTU CORE · all snaps
- Ubuntu Core with Secure Boot and a TPM — what the Secure Boot pair is for
ARCHITECTURES · machines and firmware
- Boot types — which systems boot efi at all
- Six architectures, six QEMU machines — the machines that load the firmware
RUNTIME · make rules and the CLI
- From a target name to a VM — where the firmware copies fit in the chain of files make creates
SNAP · snap-only features
- The image-garden snap — the snap, whose QEMU components carry the firmware
CONTAINER · OCI image
- Running the container in CI — the container, its own QEMU and its firmware prefixes