U-Boot

one bootloader for Ubuntu Core developers

On U-Boot boards snapd only edits an environment and unpacks kernels. The boot logic is a script in the gadget. Since UC20 the environment is a small boot.sel file on each of ubuntu-seed and ubuntu-boot or with ubootpart a raw partition holding two copies.

On the map: ubuntu core bootloaders

▶ Watch this deck — 7 scenes, ~6 min

01 snapd edits variables, the gadget's script decides

U-BOOT snapdwrites variablesunpacks kernelsenvironmentkernel folder gadgetboot scriptreads the variables, flips the status,loads the kernel UC16/18uboot.envone environment; the logicis a variable inside it UC20+boot.sel ×2 + boot.scrone small file per partition,and a script ubootpartraw partitiontwo copies,no file system
snapd writes variables and unpacks kernels; the gadget's boot script reads the variables, flips the status and loads the kernel. Three arrangements: uboot.env on UC16/18, two boot.sel files and boot.scr on UC20 and later, and a raw partition with ubootpart

With U-Boot, snapd writes boot variables and unpacks kernel files; the gadget's boot script does the rest. UC16/18 use uboot.env, UC20+ use two boot.sel files and boot.scr, and ubootpart uses a raw partition.

On your Arm board the bootloader is U-Boot, and U-Boot will happily run whatever script it's given. So snapd takes a step back and lets the gadget do the talking. snapd sets variables in an environment and unpacks kernel files into a folder. The gadget's boot script reads the one and loads from the other. You'll meet three arrangements. Ubuntu Core 16 and 18 keep one environment file, with the boot logic stored inside it as a variable. Ubuntu Core 20 and later use two small boot.sel files and a separate script. And the newer ubootpart keeps the environment in a raw partition.

02 The environment: a checksum and a list of strings

FORMAT CRC324 bytes, little-endian flag1 byte, optional kernel_status=try\0snap_kernel=arm-kernel_40.snap\0snap_try_kernel=arm-kernel_41.snap\0\0key=value strings, each ending in NUL; two NULs end the list 0xff…fill to the end saved by overwriting in place: no rename, no truncation, then a sync"to minimize the amount of writes happening on a potential FAT partition" reading: try one header sizeCRC wrong? try the other onecreating: header as in the gadget's sampleno sample: with the flag byte
A U-Boot environment as a byte strip: a 4-byte CRC32, an optional flag byte, NUL-terminated key=value strings ending with a double NUL, and 0xff fill. snapd overwrites it in place and reads either header size

A U-Boot environment is a CRC32, an optional flag byte, NUL-terminated key=value strings, a double NUL and 0xff fill. snapd overwrites it in place and accepts either header size when reading.

Before the script, the notebook it reads. U-Boot's environment is a block of a few kilobytes with a very simple shape. It starts with a checksum. Then there may be one flag byte. Then come key=value strings, each ending in a zero byte. Two zeros end the list, and 0xff fills the rest. snapd saves it by overwriting the file in place, with no rename, to keep writes on a FAT partition to a minimum. Whether the flag byte is there depends on how your board's U-Boot was built, so snapd reads with one layout and, if the checksum doesn't match, tries the other.

03 UC20: two boot.sel files and a script

UC20 LAYOUT ubuntu-seed ubuntu-boot gadgetuboot.confempty: a marker boot.scrthe boot script uboot/ubuntu/boot.selcreated by snapd at image time4096 bytessnapd_recovery_modesnapd_recovery_system uboot/ubuntu/boot.selshipped as gadget contentnext to the kernel folderskernel_statussnap_kernelsnap_try_kernel at run time/boot/uboot = ubuntu-boot/uboot/ubuntu reference read: the Raspberry Pi gadget, 20-arm64 branch, with flash-kernel's boot script
UC20 U-Boot layout: the gadget carries an empty uboot.conf marker and boot.scr; ubuntu-seed has uboot/ubuntu/boot.sel created by snapd with the mode and recovery system; ubuntu-boot has uboot/ubuntu/boot.sel shipped by the gadget with the kernel variables, seen as /boot/uboot at run time

UC20+ gadgets carry an empty uboot.conf marker and a boot.scr. snapd creates uboot/ubuntu/boot.sel on ubuntu-seed at image time; the one on ubuntu-boot is shipped by the gadget.

Mount your board's two unencrypted partitions and look for a folder called uboot/ubuntu. There's one on each, holding a file called boot.sel. The gadget's uboot.conf used to be the whole environment. Now it's an empty marker: this gadget boots through a script, boot.scr. When the image is built, snapd creates a four kilobyte boot.sel on the seed, for the mode and the recovery system. The gadget ships the one beside the kernels, because snapd installs run bootloader configuration only for GRUB. At run time it appears under /boot/uboot. This layout comes from the Raspberry Pi gadget's UC20 branch.

04 What the boot script does

BOOT.SCR load /uboot/ubuntu/boot.sel from ubuntu-seed, the partition it started fromimport snapd_recovery_mode snapd_recovery_systemno mode: install bootargs += snapd_recovery_mode=… snapd_recovery_system=… panic=-1 run modeload boot.sel from partition number + 1import snap_kernel snap_try_kernel kernel_statusby number, not by label try → trying, boot snap_try_kerneltrying → cleared, boot snap_kernelsave boot.selprefix = uboot/ubuntu/<kernel>/ any other modeprefix =systems/<label>/kernel/on ubuntu-seed load kernel.img and initrd.img from the prefix, then boot
The reference boot script: load boot.sel from ubuntu-seed and import the mode and recovery system (default install), add them to the kernel arguments, then in run mode load the next partition's boot.sel, flip try to trying or clear trying, save it and use the kernel's folder; in other modes use the recovery system's kernel folder; finally load kernel.img and initrd.img and boot

The reference boot.scr reads the mode from ubuntu-seed's boot.sel (default install); in run mode it reads the next partition's boot.sel, flips try to trying, saves it, and loads kernel.img and initrd.img from the chosen kernel's folder.

Now the script. It's short, and if you've seen the GRUB deck you'll recognise most of it. It loads boot.sel from the seed and imports the mode and the recovery system. No mode means install. Both go onto the kernel command line. In run mode it loads the other boot.sel from the next partition, by number, not by label. Try becomes trying and the try kernel is picked. Trying means the last attempt failed, so the status is cleared and the old kernel is picked. It saves the file. Other modes use the recovery system's kernel. Last, it loads the kernel and initrd, and boots.

05 A kernel refresh: a folder and one write

REFRESH ubuntu-boot: uboot/ubuntu/boot.selarm-kernel_40.snap/kernel.img initrd.img dtbs/ arm-kernel_41.snap/kernel.img initrd.img dtbs/unpacked from the new kernel snap snapd, one writesnap_try_kernel=arm-kernel_41.snapkernel_status=try boot script, during the rebootkernel_status=trying snapd, one write, after a good bootsnap_kernel=arm-kernel_41.snapsnap_try_kernel=kernel_status= each change is one write of one blockno order to get wrong recovery systems: the same files in systems/<label>/kernel/ on ubuntu-seed, also on a running device
A kernel refresh on U-Boot: snapd unpacks kernel.img, initrd.img and dtbs into uboot/ubuntu/arm-kernel_41.snap/ next to arm-kernel_40.snap/; one write sets snap_try_kernel and kernel_status=try; the boot script sets trying; after a good boot one more write sets snap_kernel and clears the other two

snapd unpacks kernel.img, initrd.img and dtbs/* into uboot/ubuntu/<snap-file>/, then sets snap_try_kernel and kernel_status=try in one write; success is one more write.

Time to refresh the kernel on your board. Compared with GRUB's careful sequence of symlinks, this one is almost boring. snapd unpacks the kernel image, the initrd and the device trees from the new snap into a folder named after the snap file. Then one write sets the try kernel and the status, try. During the reboot the script flips it to trying. After a good boot, one more write makes the new kernel current and clears the other two. Each change is a single write, so there's no symlink order to get wrong. Recovery systems get the same files under systems/<label>/kernel/, even on a running device.

06 ubootpart: the environment without a file system

UBOOTPART ubuntu-boot-staterole system-boot-state in gadget.yaml · fixed offset · no file system · at least 1 MiB copy 1 · 8192 bytes · flag 7copy 2 · 8192 bytes · flag 6 copy 1 · 8192 bytes · flag 7active: a valid CRC and the newer flag copy 1 · 8192 bytes · flag 7copy 2 · 8192 bytes · flag 8← writtena save writes only the other copy, with the flag one higher; one intact copy remains how snapd finds the partition1 · the disk the firmware booted from2 · kernel argument snapd_system_disk3 · the partition label ubuntu-boot-state one environment for bothroles, seed and bootno published gadget orreference boot script found
ubootpart: a raw partition ubuntu-boot-state with the role system-boot-state holds two 8192-byte environment copies; the copy with a valid CRC and the newer flag is active, and a save writes the other copy with the flag one higher. snapd finds the partition by the firmware's boot disk, then the snapd_system_disk kernel argument, then the partition label

ubootpart stores two 8192-byte environment copies in a raw partition labelled ubuntu-boot-state (role system-boot-state). The copy with a valid CRC and the newer flag is active; a save writes the other one.

A file on a FAT partition is a fragile place to keep what a board needs to boot. ubootpart skips the file system. The gadget declares a raw partition with the role system-boot-state: fixed offset, no file system, at least one mebibyte. Inside are two copies of the environment, eight kilobytes each. The active one has a valid checksum and the newer flag. A save writes only the other copy, flag one higher, so one intact copy always remains. snapd finds the partition by the disk the firmware booted from, then a kernel argument, then the label. No published gadget or boot script for it was found.

07 Where to go from here

ROADS U-Boot Bootloadersone interface, six of them Try, trying, donewhat the statuses mean piboot and trybootwhat the Pi uses now Little Kernelraw partitions again Model, gadget and kernelwhere boot.scr comes from Ubuntu Corethe overview
Roads from U-Boot: the bootloader interface, try and rollback, piboot on the Raspberry Pi, Little Kernel, the model and gadget, and the Ubuntu Core overview

Roads: the bootloader interface, try and rollback, piboot, Little Kernel, the gadget and the overview.

On a U-Boot board most of the boot story is written in the gadget. Here's where the rest of it is. The bootloaders deck shows the interface snapd's two U-Boot implementations share, and how it tells them apart. Try, trying, done explains what those statuses mean for the whole device. The Raspberry Pi has moved from U-Boot to piboot, and that deck shows how. Little Kernel boards keep their environment in raw partitions too. The model deck covers the gadget that carries boot.scr and declares the boot state partition. And the overview takes you back to Ubuntu Core as a whole.

Read more

Three arrangements

ArrangementEnvironmentBoot logicKernels
UC16/18 (uboot, sole role)/boot/uboot/uboot.env, a copy of the gadget's uboot.confthe variable snappy_boot inside the environment, using snap_mode/boot/uboot/<snap-file>/
UC20+ (uboot)uboot/ubuntu/boot.sel on ubuntu-seed and on ubuntu-bootthe gadget's boot.scruboot/ubuntu/<snap-file>/ on ubuntu-boot; systems/<label>/kernel/ on ubuntu-seed
ubootpartraw partition ubuntu-boot-state, two copiesa gadget boot script (no reference found)uboot/ubuntu/<snap-file>/; systems/<label>/kernel/

The gadget.yaml value is u-boot; snapd's names are uboot and ubootpart; the marker file is uboot.conf. On UC20 and later a non-empty uboot.conf is refused with "non-empty uboot.env not supported on UC20+ yet".

Environment format

PartSizeNotes
CRC324 byteslittle-endian
flag1 byteoptional in a single-copy environment; always present in a redundant one
datarestkey=value strings ending in NUL; a double NUL ends the list; 0xff fill

When snapd creates boot.sel it takes only the header layout, flag byte or not, from a sample boot.sel in the gadget if there is one; the content is not copied. Without a sample it writes the flag byte.

File or deviceSize
boot.sel created by snapd4096 bytes
UC16/18 uboot.env in the Pi gadget131072 bytes
each copy on ubuntu-boot-state8192 bytes
ubuntu-boot-state.img written at image build time16384 bytes

Variables

FileVariables
ubuntu-seed boot.selsnapd_recovery_mode, snapd_recovery_system, snapd_recovery_kernel
ubuntu-boot boot.selkernel_status, snap_kernel, snap_try_kernel
UC16/18 uboot.envsnap_mode, snap_core, snap_try_core, snap_kernel, snap_try_kernel

snapd does not write the kernel command line variables for U-Boot; the boot script owns the command line.

The reference boot script, step by step

The script read is flash-kernel's bootscr.rpi, which the Raspberry Pi gadget's 20-arm64 branch wraps into boot.scr.

1. Load /uboot/ubuntu/boot.sel from the partition U-Boot booted from (ubuntu-seed); import the three recovery variables. The default mode is install. 2. Append snapd_recovery_mode=... snapd_recovery_system=... panic=-1 to the kernel arguments. 3. In run mode: load boot.sel from the next partition (partition number plus one), import the three kernel variables, flip try to trying or clear trying, save the file, and set the prefix to uboot/ubuntu/<kernel>/. 4. Otherwise: set the prefix to systems/<snapd_recovery_system>/kernel/. 5. Load kernel.img (decompressing it if needed) and initrd.img, then boot.

ubootpart in gadget.yaml

A structure with role: system-boot-state must have an explicit offset, no min-size, a size of at least 1 MiB, no file system, and the type 3DE21764-95BD-54BD-A5C3-4ABE786F38A8. The volume says bootloader: u-boot. A gadget reference file ubootpart.sel, if present, must be 8192 bytes.

Partition lookup order: the disk of the partition the firmware reports as the loader's; the disk named by the kernel argument snapd_system_disk; /dev/disk/by-partlabel/ubuntu-boot-state.

Further reading

Roads from here

UBUNTU CORE · boot

UBUNTU CORE · bootloaders

UBUNTU CORE · install and recovery