Bootloaders: one interface, six implementations
the shared mechanism for Ubuntu Core developers for snapd developers
snapd talks to every bootloader through one small interface for reading and writing boot variables and placing kernels. Six implementations are probed in a fixed order. Only GRUB has trusted assets and a managed configuration and command line which is why only GRUB supports TPM-backed encryption.
On the map: ubuntu core bootloaders
01 Three things snapd needs from any bootloader
The Bootloader interface has eight methods: get and set boot variables, extract and remove kernel assets, install the boot config, name, presence and whether a gadget requires it. Optional interfaces describe everything else.
Put four boards on the bench: an x86 gateway, a Raspberry Pi, an Arm board with U-Boot and a Dragonboard. Their boot code has nothing in common, and snapd has to refresh a kernel on all four. So what does snapd need from a bootloader? Not much. It needs it to remember a few named variables. It needs to put a kernel's boot files where the bootloader can load them, and take them away again. And it needs to set the bootloader up from the gadget. Add a name and a presence check, and that's eight methods. Everything else is optional, and that's where the six implementations part ways.
02 Which bootloader is this?
Find probes ubootpart, uboot, grub, androidboot, lk and piboot in that order and takes the first one present. ForGadget looks for a marker file <name>.conf in the gadget; ubootpart is recognised from gadget.yaml.
snapd on the gateway has no setting that says this is a GRUB machine. It works it out each time. On a running device it asks each implementation whether it's present, in a fixed order, and the first yes wins. ubootpart is asked before U-Boot on purpose. If one is present but broken, the search stops with an error, so a later one can't win. When snapd starts from a gadget, to build an image or install, it looks for an empty marker file named after the implementation. It's uboot.conf, without the hyphen gadget.yaml uses. ubootpart has no marker; gadget.yaml alone identifies it.
03 Three roles
Roles: recovery on ubuntu-seed, run-mode on ubuntu-boot, and the sole bootloader of UC16/18. The run bootloader is created in install mode, not at image build time.
The gateway has two bootloaders, and they don't do the same job. The one on ubuntu-seed has the recovery role. It picks the mode and the recovery system. The one on ubuntu-boot has the run-mode role, and it decides which kernel to start. The recovery one is put in place when the image is built. The run one only appears in install mode, and the code refuses to set it up at image time. Older releases had a single bootloader, in the sole role. And snap debug boot-vars prints the run variables, or the recovery ones when you point it at the seed.
04 A dozen variables carry everything
Recovery variables: snapd_recovery_mode, snapd_recovery_system, try_recovery_system, recovery_system_status. Run variables: kernel_status, snap_kernel, snap_try_kernel and the two command-line variables.
If you had to tell a bootloader what to do next using only a handful of words, which would you pick? snapd picked about a dozen. On the recovery side there's the mode, and the label of the recovery system to boot. Two more are used while a new recovery system is on trial. On the run side there's the kernel_status, which moves through empty, try and trying. on-bootloaders without symlinks, two more name the current and the try kernel. Two carry the kernel command line, and only GRUB uses them. On GRUB, each recovery system also names its kernel. The full table is below the deck.
05 Two ways to hold a kernel
GRUB keeps the current and try kernels as symlinks (kernel.efi, try-kernel.efi) plus kernel_status. Other bootloaders use snap_kernel, snap_try_kernel and kernel_status, set in a single write. snapd picks the strategy by checking for ExtractedRunKernelImageBootloader.
Back to the bench. The same kernel refresh is about to land on all four boards, and snapd records it in one of two ways. GRUB keeps the current and the try kernel as two symlinks next to grub.cfg, and the status as a variable. So on the gateway a refresh is separate writes, and their order matters: the try symlink goes first, then the status. Every other bootloader keeps all three facts as variables. Setting up the try is one write of the environment, and so is committing it. The code notes that ordering doesn't matter there, because there's only one mutation.
06 Who flips try to trying?
GRUB: snapd's managed grub.cfg. U-Boot: the gadget's boot.scr. lk: C code in the bootloader. piboot: snap-bootstrap in the initramfs. Every board flips try to trying and treats trying at boot as a failure; only GRUB also clears an invalid value. The initramfs check of the try kernel runs on top on every board.
Someone has to change try into trying before the new kernel starts. On our four boards it's four different someones. On the gateway, it's the grub.cfg that snapd itself ships. On the Arm board, it's the gadget's boot.scr script. On the Dragonboard, it's C code compiled into the bootloader. The Pi's firmware can't run a script or write a variable, so snapd marks the try on the kernel command line, and snap-bootstrap flips it from the initramfs. Wherever it lives, try becomes trying, and trying at boot means it failed. For ubootpart and Android boot, no boot-side code was found.
07 Trusted assets, managed assets, and why only GRUB
Trusted assets are boot binaries whose hashes snapd tracks in modeenv and caches under /var/lib/snapd/boot-assets; managed assets are files snapd writes itself. Only GRUB is a TrustedAssetsBootloader, so only GRUB has boot chains and a managed command line.
Three of the four boards can't do TPM-backed disk encryption. Even with a TPM, the bootloader would decide. A trusted asset is a boot binary whose exact content matters, like shim or GRUB. When a gadget refresh replaces one, snapd records old and new hashes in modeenv, keeps a copy of each version, and after a good boot drops the hash it no longer needs. A managed asset is a file snapd writes itself, like grub.cfg, and a gadget update leaves it alone. Only GRUB has either. So only on GRUB can snapd describe the chain for sealing keys to the TPM, and manage the command line.
08 Where to go from here
Roads: one deck per bootloader, then try and rollback, the command line, boot chains and the overview.
Each board on the bench has its own deck, so pick the one on your desk. The GRUB deck has two stages and a configuration that snapd manages. The Raspberry Pi deck is about a bootloader that can't write anything itself. The U-Boot deck has a script from the gadget and an environment that can live in a raw partition. The Little Kernel deck has no file system at all. Then three decks follow the mechanisms: try, trying and rollback; the kernel command line; and sealed keys with their boot chains. Or go back to the Ubuntu Core overview.
Read more
The implementations
| snapd name | gadget.yaml bootloader: | Marker in the gadget | Used on |
|---|---|---|---|
grub | grub | grub.conf | x86-64 and arm64 UEFI |
piboot | piboot (UC20 onwards) | piboot.conf | Raspberry Pi, firmware loads the kernel |
uboot | u-boot | uboot.conf | U-Boot with the environment in a file |
ubootpart | u-boot plus a structure with role: system-boot-state | none | U-Boot with the environment in a raw partition |
lk | lk | lk.conf | Little Kernel (aboot) on Qualcomm boards |
androidboot | android-boot | androidboot.conf | Android boot; not usable as a UC20+ recovery bootloader |
Probe order at runtime: ubootpart, uboot, grub, androidboot, lk, piboot. ForGadget walks the same list, so a gadget with uboot.conf and a system-boot-state structure gets ubootpart.
Optional interfaces
| Interface | Meaning | grub | piboot | uboot | ubootpart | lk | androidboot |
|---|---|---|---|---|---|---|---|
RecoveryAwareBootloader | per-recovery-system environment | yes | no | no | no | no | no |
ExtractedRecoveryKernelImageBootloader | recovery kernels are unpacked | no | yes | yes | yes | image time only | no |
ExtractedRunKernelImageBootloader | run kernels as symlinks | yes | no | no | no | no | no |
TrustedAssetsBootloader | trusted assets, managed config, command line, boot chains | yes | no | no | no | no | no |
UefiBootloader | EFI boot entries | yes (recovery role) | no | no | no | no | no |
NotScriptableBootloader | the initramfs flips the status | no | yes | no | no | no | no |
RebootBootloader | needs an argument to the reboot call | no | yes | no | no | no | no |
RecoveryBootConfigBootloader | config can be rebuilt after a gadget update | no | yes | no | no | no | no |
A recovery bootloader must unpack recovery kernels or be recovery-aware; otherwise: "cannot use %s bootloader: does not support recovery systems".
Boot variables
| Variable | Role | Meaning |
|---|---|---|
snapd_recovery_mode | recovery | mode of the next boot; unset means install |
snapd_recovery_system | recovery | label of the recovery system to use |
snapd_recovery_kernel | per recovery system, GRUB only | the kernel snap of that system; other bootloaders unpack the kernel into the system directory |
try_recovery_system | recovery | label of a recovery system being tested |
recovery_system_status | recovery | try, then tried |
snapd_good_recovery_systems | recovery, GRUB only | systems that passed their test |
snapd_boot_flags | recovery | flags for the next boot |
kernel_status | run | empty, try or trying |
snap_kernel | run | current kernel, on bootloaders without symlinks |
snap_try_kernel | run | kernel being tried, on bootloaders without symlinks |
snapd_extra_cmdline_args | run and per recovery system, GRUB only | arguments appended to the static command line |
snapd_full_cmdline_args | run and per recovery system, GRUB only | a complete replacement of the static command line |
snap_mode, snap_core, snap_try_core | sole (UC16/18) | status and base of the older scheme |
snap debug boot-vars prints them; --root-dir /run/mnt/ubuntu-seed selects the recovery bootloader.
Who changes try to trying
| Bootloader | Who | Where |
|---|---|---|
| GRUB, UC20+ | snapd's managed grub.cfg | part of snapd |
| GRUB, UC16/18 | the gadget's grub.cfg (snap_mode) | pc gadget |
| piboot | snap-bootstrap in the initramfs | part of snapd |
| U-Boot, UC20 | the gadget's boot.scr | built from flash-kernel's Raspberry Pi boot script |
| U-Boot, UC16/18 | snappy_boot in the gadget's built-in environment (snap_mode) | pi gadget |
| ubootpart | a gadget boot script | not determined: no reference script was found |
| lk, UC20 | a C function in the lk fork | snap_get_target_boot_params |
| lk, UC16/18 | a patch to aboot in the gadget (snap_mode) | dragonboard gadget |
| androidboot | not determined | no boot code was found |
Where the environment is kept, and how it is protected
| Bootloader | Storage | Integrity |
|---|---|---|
| GRUB | grubenv, a 1024-byte file | none |
| piboot | piboot.conf, U-Boot environment format | CRC32 |
| uboot | boot.sel or uboot.env file | CRC32 |
| ubootpart | raw partition ubuntu-boot-state | CRC32 and a second copy |
| lk | raw partitions snapbootsel, snaprecoverysel | CRC32 and a backup partition |
| androidboot | androidboot.env, a text file | none |
Trusted and managed assets
- Trusted assets are named in
current_trusted_boot_assets(run bootloader) andcurrent_trusted_recovery_boot_assets(recovery bootloader) in modeenv, each with one or two SHA3-384 hashes. - Every trusted version is copied to
/var/lib/snapd/boot-assets/<bootloader>/<name>-<hash>. - After a successful boot the list is trimmed to the hash that booted. An unexpected hash is an error only when the device is encrypted.
- A managed asset is replaced only when snapd's built-in copy has a higher
Snapd-Boot-Config-Edition.
Further reading
- Gadget snap format: the
bootloader:key, the marker files and the bootloader-related roles. - Build a gadget snap: where GRUB or U-Boot assets sit in a gadget.
- Updating gadget boot assets:
update.edition,preserveand rollback of boot assets on a gadget refresh. - Storage layout: the first-stage and second-stage bootloader.
- Board enablement: a worked U-Boot board from the UC16 era, useful for the contrast with UC20.
Roads from here
UBUNTU CORE · boot
- Try, trying, done — the try states these variables carry
- The kernel command line — the command line that only GRUB lets snapd manage
- Ubuntu Core: how a device boots and recovers — back to the Ubuntu Core overview
UBUNTU CORE · bootloaders
- GRUB on Ubuntu Core — GRUB with its two stages and managed configuration
- Raspberry Pi: piboot and tryboot — the Raspberry Pi firmware and tryboot
- U-Boot — U-Boot with boot.sel or a raw environment partition
- Little Kernel and Android boot — Little Kernel and Android boot
UBUNTU CORE · install and recovery
- Model, gadget and kernel — the gadget that names the bootloader
UBUNTU CORE · full disk encryption
- Sealed keys, boot chains and resealing — boot chains built from trusted assets