GRUB on Ubuntu Core

one bootloader for Ubuntu Core developers

On UEFI devices the gadget ships signed shim and GRUB binaries and snapd ships both GRUB configuration files. A GRUB on ubuntu-seed chooses the mode and a GRUB on ubuntu-boot chooses the kernel through two symlinks. State is kept in three grubenv files and snapd replaces its own grub.cfg when a newer edition is built in.

On the map: ubuntu core bootloaders

▶ Watch this deck — 8 scenes, ~6 min

01 The gadget brings the binaries, snapd brings the script

WHO SHIPS WHAT pc gadgetshimx64.efigrubx64.efigrub.confempty: a markersigned snapdgrub-recovery.cfggrub.cfgbuilt in ubuntu-seedshim · grubx64.efiubuntu-bootgrubx64.efi EFI/ubuntu/grub.cfgEFI/ubuntu/grub.cfg gadget.yaml: "bootloader configuration is shipped and managed by snapd" UC16 and UC18: the gadget shipped its own grub.cfg
The pc gadget snap carries shimx64.efi, grubx64.efi and an empty grub.conf marker; snapd carries grub-recovery.cfg and grub.cfg; the binaries land on ubuntu-seed and ubuntu-boot and snapd writes each script as EFI/ubuntu/grub.cfg on its partition

The gadget ships signed shim and GRUB binaries and an empty marker grub.conf. snapd writes EFI/ubuntu/grub.cfg on both partitions from its built-in grub-recovery.cfg and grub.cfg.

Open the gateway's gadget snap and look for the GRUB configuration. You won't find one. There's an empty file called grub.conf, and all it says is: this device uses GRUB. Next to it sit the signed shim and GRUB binaries, built from Ubuntu's packages. The scripts come from snapd. It has two built in, one for each stage, and writes them as grub.cfg onto the seed and boot partitions. That's why the logic of modes and kernel tries can change with a snapd refresh, not a gadget refresh. On Ubuntu Core sixteen and eighteen it was the other way round: the gadget shipped the script.

02 What sits on each partition

FILES ubuntu-seedthe EFI system partitionEFI/boot/bootx64.efishimEFI/boot/fbx64.efifallback EFI/ubuntu/shimx64.efishimEFI/ubuntu/grubx64.efifirst GRUB EFI/ubuntu/grub.cfgEFI/ubuntu/grubenvsystems/20260301/grubenvone per system ubuntu-bootEFI/boot/grubx64.efisecond GRUBEFI/ubuntu/grub.cfgEFI/ubuntu/grubenv EFI/ubuntu/pc-kernel_2901.snap/kernel.efiextractedEFI/ubuntu/kernel.efi-> pc-kernel_2901.snap/kernel.efi what shim verifies and measures: outside snapdsnapd hashes these and sets the boot entry "ubuntu"
File listings of ubuntu-seed and ubuntu-boot: shim twice, the first GRUB, grub.cfg, grubenv and a grubenv per recovery system on the seed; a second GRUB, grub.cfg, grubenv, an extracted kernel.efi and a kernel.efi symlink on boot

ubuntu-seed holds shim, the first GRUB, EFI/ubuntu/grub.cfg, grubenv and a grubenv per recovery system. ubuntu-boot holds a second GRUB, its config and environment, and one extracted kernel.efi per kernel snap.

Let's mount the gateway's two unencrypted partitions and look. The seed partition is the EFI system partition. It has shim in the firmware's default place, and again under EFI/ubuntu, next to the first GRUB, its script and its grubenv. Each recovery system gets a small grubenv of its own. The boot partition has a second GRUB, its own script and environment, and a folder per kernel snap holding one file, kernel.efi, with a symlink to the current one. What shim checks and measures is outside snapd. snapd knows these files by name, hashes them, and points the firmware's boot entry at shim.

03 Three small files hold the state

GRUBENV ubuntu-seedEFI/ubuntu/grubenvsnapd_recovery_mode=runsnapd_recovery_system=20260301…and snapd's ownwhich mode, which system ubuntu-seedsystems/20260301/grubenvsnapd_recovery_kernel=/snaps/pc-kernel_2901.snapsnapd_extra_cmdline_args=this system's kernel ubuntu-bootEFI/ubuntu/grubenvkernel_status=snapd_extra_cmdline_args=snapd_full_cmdline_args=kernel try, command line # GRUB Environment Blockkernel_status=##########################################################################################…exactly 1024 bytes no checksum, no second copy: overwritten in place
Three grubenv files: on ubuntu-seed with the recovery mode and system, in systems/20260301 with the recovery kernel, and on ubuntu-boot with kernel_status and the command line; each is a 1024-byte block with a header, key=value lines and hash padding, with no checksum

Three grubenv files: on ubuntu-seed for the mode and recovery system, in each systems/<label> for that system's kernel and command line, and on ubuntu-boot for kernel_status and the command line. Each is exactly 1024 bytes with no checksum; snapd truncates and rewrites it in place.

GRUB can read files and save a few variables. On the gateway, everything it remembers between boots lives in three files, each exactly a kilobyte. The one on seed answers two questions: which mode, and which recovery system. snapd keeps a few extras there too. Each recovery system has its own, naming that system's kernel snap and command line. The one on-boot holds the kernel_status and the run command line. The format is GRUB's: a header, key=value lines, and hash characters as padding. There's no checksum and no second copy, and snapd overwrites the file in place, so it doesn't move on disk.

04 A kernel refresh, seen as files

REFRESH ubuntu-boot · EFI/ubuntu/ pc-kernel_2901.snap/kernel.efikernel.efi -> pc-kernel_2901.snap/kernel.efi pc-kernel_2950.snap/kernel.efiextracted from the new snap try-kernel.efi -> pc-kernel_2950.snap/kernel.efiubuntu-boot grubenvkernel_status=try after a good bootkernel.efi -> pc-kernel_2950.snap/kernel.efistatus cleared relative symlinks, replaced atomicallya try link to a kernel that was not extracted is refusedwhy the order is safe: try and rollback
The EFI/ubuntu folder on ubuntu-boot during a kernel refresh: kernel.efi from pc-kernel_2950 is extracted, a try-kernel.efi symlink and kernel_status=try are added, and after a good boot kernel.efi points at the new kernel and try-kernel.efi is removed

snapd extracts kernel.efi into EFI/ubuntu/<snap-file>/ on ubuntu-boot. kernel.efi and try-kernel.efi are relative symlinks into those folders, replaced atomically.

Here's the gateway's kernel refresh again. Watch one folder on the boot partition. The run GRUB starts before anything has unlocked the data partition, so the kernel can't come from there. Instead, snapd extracts one file, kernel.efi, into a folder named after the new snap file. The try is a second symlink, try-kernel.efi, plus the kernel_status set to try. After a good boot, snapd points the first symlink at the new kernel and removes the second. Both links are relative and swapped atomically, and snapd refuses a try link to a kernel it hasn't extracted. The try deck explains why that order is safe.

05 The fallback entry reboots

FALLBACK 0 · Run Ubuntu Corechainloader $prefix/try-kernel.efi ... set kernel_status="trying"save_env kernel_statusset fallback=1all before the chainload cannot start1 · Fallback on failed updateecho "Cannot start new kernel - booting previous one"reboot TPMmeasurements already takenmust start afresh to unseal rebootkernel_status=tryingcleared, boots kernel.efi a kernel that will not even load: one extra reboot
The run grub.cfg saves kernel_status=trying and sets fallback=1, then fails to chainload try-kernel.efi; the fallback entry prints a message and reboots, because the TPM measurements must start afresh; the next boot finds trying, clears it and boots kernel.efi

With kernel_status=try GRUB saves trying and sets fallback=1. The fallback entry prints "Cannot start new kernel - booting previous one" and reboots, because the TPM measurements of the failed attempt would prevent unsealing the key.

Suppose the new kernel is so broken GRUB can't even start it. The old kernel's right there in the next folder. Why not just load that? When the status is try, the script saves trying and sets GRUB's fallback to the second menu entry, and only then chainloads the try kernel. If that fails, the second entry runs. It prints a message and reboots. The script's comment says why: the TPM measurements have to be cleaned up before the key can be unsealed. A reboot starts them afresh. GRUB finds trying, clears it and boots the old kernel. So a kernel that won't even load costs one extra reboot.

06 The edition header

MANAGED CONFIG on the gateway's ubuntu-boot# Snapd-Boot-Config-Edition: 3set default=0 ... built into the refreshed snapd# Snapd-Boot-Config-Edition: 3set default=0 ...update-managed-boot-config 3 = 3: nothing happens an older deviceedition 2 < 3replace grub.cfgrebootcommand line recordedand resealed first no edition header on disk: not managed, left alone only the run config is updatedgrub.cfg on ubuntu-seed stays as installed: "not supported yet"
The grub.cfg on the gateway's ubuntu-boot and the one built into a refreshed snapd both start with Snapd-Boot-Config-Edition 3, so nothing happens; on an older device edition 2 is lower than 3, so the file is replaced and the device reboots; a file without the header is left alone, and the recovery config is not updated

A managed grub.cfg starts with # Snapd-Boot-Config-Edition: N. A snapd refresh replaces the run config only when its built-in edition is higher, then reboots. The recovery config on ubuntu-seed is not updated at runtime.

snapd wrote the gateway's GRUB script at install time, and snapd has been refreshed many times since. Is the script still the factory one? Each built-in script starts with a comment carrying an edition number. After every snapd refresh, a task compares the edition on disk with its own. On the gateway both say three, so nothing happens. On an older device with edition two, the file's replaced and the device reboots. A new edition can change the static command line, so snapd records that first and reseals. A file without the header is left alone. And only the run script is updated; the recovery side isn't supported yet.

07 The recovery menu

SEED MENU GRUB on ubuntu-seed Continue to run modechainloads the GRUB on ubuntu-boot Recover using 20260301Install using 20260301Factory reset using 20260301three entries per folder in /systems UEFI Firmware Settings Continue to run modedefault: the stored mode and systemon the gateway: run · hidden · 3 s timeout snapd_recovery_mode=runsaved first: recover is one-shot recovery kernel: not extracted; loop-mounted from the snap named insystems/20260301/grubenv snapd_recovery_kernel=/snaps/pc-kernel_2901.snap
The menu of the GRUB on ubuntu-seed: Continue to run mode, then Recover, Install and Factory reset using 20260301, then UEFI Firmware Settings; the default follows the stored mode and system, the menu is hidden with a 3 second timeout, Recover first saves snapd_recovery_mode=run, and the recovery kernel is loop-mounted from the seed

The seed GRUB lists "Continue to run mode", then Recover, Install and Factory reset for each system under /systems, and "UEFI Firmware Settings". The default is the stored mode and system; the Recover entry first saves snapd_recovery_mode=run.

Connect a screen and a keyboard to the gateway, stop GRUB early enough, and you get a menu most owners never see. The first GRUB offers to continue to run mode, if it can find the boot partition. Then, for every folder under systems that looks like a label, it adds recover, install and factory-reset. Last comes the firmware settings. The menu's hidden, with a three second timeout, and the default is the stored mode and system: on the gateway, run. Recover writes run back before starting the kernel, so it's a one-shot. And recovery kernels aren't extracted: GRUB loop-mounts the kernel snap named in that system's grubenv.

08 Where to go from here

ROADS GRUB on Ubuntu Core Bootloaderswhat the other five lack From power-on to snapdboth GRUB stages Try, trying, donewhat the statuses mean The kernel command linehow the variables are filled Sealed keys, boot chainswhy shim and GRUB hashes matter Recovery systemswhat the seed menu lists Ubuntu Corethe overview
Roads from GRUB on Ubuntu Core: bootloaders, from power-on to snapd, try and rollback, the kernel command line, sealed keys and boot chains, recovery systems, and the Ubuntu Core overview

Roads: the bootloader interface, the boot path, try and rollback, the command line, boot chains, recovery systems and the overview.

GRUB is the bootloader snapd knows best, and the only one it writes the script for. Here's where its neighbours are. The bootloaders deck has the interface GRUB implements more of than any other. The boot path walks a whole run-mode boot through both GRUB stages. Try and rollback explains what try and trying mean. The command line deck shows how those variables get filled. Sealing explains why the hashes of shim and GRUB matter. Recovery systems covers the systems this menu lists. And the Ubuntu Core overview is one step back.

Read more

Files

PartitionPathWhat it is
ubuntu-seedEFI/boot/bootx64.efishim, in the firmware's default location
ubuntu-seedEFI/boot/fbx64.efishim's fallback program; not part of a trusted chain
ubuntu-seedEFI/ubuntu/shimx64.efishim
ubuntu-seedEFI/ubuntu/grubx64.efithe first GRUB
ubuntu-seedEFI/ubuntu/grub.cfgsnapd's grub-recovery.cfg
ubuntu-seedEFI/ubuntu/grubenvmode and recovery system
ubuntu-seedsystems/<label>/grubenvthat system's kernel and command line
ubuntu-bootEFI/boot/grubx64.efithe second GRUB
ubuntu-bootEFI/ubuntu/grub.cfgsnapd's grub.cfg
ubuntu-bootEFI/ubuntu/grubenvkernel status and command line
ubuntu-bootEFI/ubuntu/<snap-file>/kernel.efian extracted kernel
ubuntu-bootEFI/ubuntu/kernel.efi, try-kernel.efisymlinks to the current and the try kernel

On a running device EFI/ubuntu of ubuntu-boot is also visible as /boot/grub, and ubuntu-seed as /boot/efi. arm64 uses grubaa64.efi. Architectures other than amd64 and arm64 have no GRUB assets in snapd.

The three grubenv files

FileVariablesRead by
ubuntu-seed EFI/ubuntu/grubenvsnapd_recovery_mode, snapd_recovery_system, try_recovery_system, recovery_system_status, snapd_good_recovery_systems, snapd_boot_flagsthe seed GRUB loads the first two; the rest are for snapd
ubuntu-seed systems/<label>/grubenvsnapd_recovery_kernel, snapd_extra_cmdline_args, snapd_full_cmdline_argsthe seed GRUB
ubuntu-boot EFI/ubuntu/grubenvkernel_status, snapd_extra_cmdline_args, snapd_full_cmdline_argsthe run GRUB

Format: exactly 1024 bytes; first line # GRUB Environment Block; key=value lines; padded with #. A file of another size is rejected: "grubenv %q must be exactly 1024 byte, got %d".

snapd_recovery_kernel is the kernel snap's path inside ubuntu-seed, for example /snaps/pc-kernel_2901.snap. An unasserted kernel on a dangerous model is under /systems/<label>/snaps/. After an install from recovery system 20260301 the seed environment holds snapd_recovery_mode=run and snapd_recovery_system=20260301.

What the run grub.cfg does with kernel_status

Value at bootAction
trysave trying, set fallback=1, boot try-kernel.efi
tryingclear it, boot kernel.efi
any other non-empty valueprint "invalid kernel_status!!!", clear it, boot kernel.efi
emptyboot kernel.efi

Editions and static command lines

ConfigEditionStatic arguments on x86-64On arm64
grub.cfg (run)3console=ttyS0,115200n8 console=tty1 panic=-1panic=-1
grub.cfg (run), editions 1 and 21, 2console=ttyS0 console=tty1 panic=-1panic=-1
grub-recovery.cfg2console=ttyS0 console=tty1 panic=-1panic=-1

A grub.cfg without the edition header is not updated, and is treated as edition 1 when the command line is computed. The task update-managed-boot-config is added to every refresh of the snapd snap on Ubuntu Core; it replaces the run grub.cfg only when the built-in edition is higher, and requests a system restart when it changed the file or the command line.

Menu entries of the seed GRUB

EntryHotkeyWhat it does
Continue to run modenchainloads the GRUB on ubuntu-boot
Recover using <label>rsaves snapd_recovery_mode=run, then boots the system's kernel in recover mode
Install using <label>iboots the system's kernel in install mode
Factory reset using <label>iboots the system's kernel in factory-reset mode
UEFI Firmware Settingsfenters the firmware set-up

Install and Factory reset are both given the hotkey i in the configuration. With no mode stored, the default mode is install; with no system stored, the lowest label is used. "Continue to run mode" is only offered when a partition labelled ubuntu-boot is found. Hybrid classic systems use grub-recovery-hybrid.cfg, which offers one entry for the mode and system snapd selected, and defaults to run mode when that system is missing.

Trusted assets and shim updates

  • Recovery chains accepted: EFI/ubuntu/shimx64.efi then EFI/ubuntu/grubx64.efi, or EFI/boot/bootx64.efi then EFI/boot/grubx64.efi.
  • The run chain adds EFI/boot/grubx64.efi from ubuntu-boot and ends in kernel.efi.
  • When a successful boot drops an old shim hash from modeenv, the next reseal also revokes the old TPM policies, so an older shim cannot be put back to reuse old sealed keys.
  • snapd writes the firmware boot entry with the description ubuntu, pointing at EFI/ubuntu/shimx64.efi, or at EFI/boot/bootx64.efi when only that one is installed.

Further reading

Roads from here

UBUNTU CORE · boot

UBUNTU CORE · bootloaders

UBUNTU CORE · install and recovery

UBUNTU CORE · full disk encryption