Raspberry Pi: piboot and tryboot

one bootloader for Ubuntu Core developers

The Raspberry Pi firmware loads the kernel itself and can neither run a script nor save a variable. snapd keeps its boot variables in piboot.conf on ubuntu-seed and rewrites the firmware's config.txt from them. A new kernel is tried through tryboot.txt and a reboot argument and the initramfs changes try to trying.

On the map: ubuntu core bootloaders

▶ Watch this deck — 8 scenes, ~6 min

01 A bootloader that cannot write

PIBOOT Pi firmware (EEPROM)no scriptno saved variables config.txtreads snapdwrites piboot.confsnapd's own variables;the firmware cannot read it snap-bootstrapin the initramfs:does the try to trying flip piboot is snapd's name for this arrangement; there is no program called piboot
The Pi firmware, which cannot script or save variables, reads config.txt; snapd writes config.txt from its own piboot.conf, which the firmware cannot read; snap-bootstrap in the initramfs changes try to trying

The Pi firmware reads config.txt and loads a kernel; it cannot script or write. snapd keeps boot variables in piboot.conf, regenerates the firmware's files from them, and snap-bootstrap flips the kernel status in the initramfs.

There's a Raspberry Pi 4 on your desk, running Ubuntu Core. Its firmware is the least helpful bootloader you could ask for: it can't run a script, and it can't save a variable. It reads config.txt and loads the kernel that file names. So snapd turns the roles round. It keeps its boot variables in its own file, piboot.conf, which the firmware can't read, and whenever they change it rewrites the firmware's configuration to match. The one job that must happen during boot, changing try to trying, moves later, into snap-bootstrap in the initramfs. And piboot? That's snapd's name for this arrangement.

02 Everything lives on ubuntu-seed

ONE PARTITION ubuntu-bootno bootloader files ubuntu-seedpiboot/ubuntu/piboot.conf config.txt cmdline.txtstart* fixup* bootcode* piboot/ubuntu/pi-kernel_812.snap/kernel.img initrd.img *.dtb overlays/ systems/20260301/kernel/kernel.img initrd.img *.dtb overlays/a recovery system's kernel, unpacked /boot/piboot = ubuntu-seed/piboot/ubuntuas seen on the running system
Two partitions: ubuntu-seed holds piboot.conf, config.txt, cmdline.txt, the firmware files, the unpacked run kernel pi-kernel_812.snap and the recovery kernel under systems; ubuntu-boot holds no bootloader files; on the running system the piboot folder appears as /boot/piboot

On a Pi both roles use piboot/ubuntu/piboot.conf on ubuntu-seed. Run kernels are unpacked to piboot/ubuntu/<snap-file>/ and recovery kernels to systems/<label>/kernel/, also on ubuntu-seed.

On the x86 gateway, the run bootloader lives on ubuntu-boot. On the Pi, go and look at ubuntu-boot: there's nothing there. The firmware reads from the first partition, and that's ubuntu-seed. So piboot.conf sits on seed, in the piboot/ubuntu folder, whichever role snapd asks for. The gadget's config.txt, command line file and firmware sit at the top of seed. Run kernels are unpacked there too, each into a folder named after its snap file: kernel image, initrd, device trees and overlays. A recovery system's kernel goes under systems, its label, kernel. On the running Pi, that piboot folder shows up at /boot/piboot.

03 One line in config.txt chooses the kernel

OS_PREFIX config.txt, from the gadgetkernel=kernel.imgcmdline=cmdline.txtinitramfs initrd.img followkernelos_prefix= piboot.confsnapd_recovery_mode=runsnap_kernel=pi-kernel_812.snapkernel_status= SetBootVarsrewritesos_prefix=/piboot/ubuntu/pi-kernel_812.snap/ pi-kernel_812.snap/cmdline.txt<gadget's first line> snapd_recovery_mode=run recover or install modeos_prefix=/systems/<label>/kernel/+ snapd_recovery_system=<label> rewritten only when snapd_recovery_mode or kernel_status changes,and again after a gadget refresh replaces config.txt
snapd's variables in piboot.conf are turned into config.txt with os_prefix set to the folder of pi-kernel_812.snap and into a cmdline.txt in that folder that adds snapd_recovery_mode=run; in other modes the prefix points at the recovery system's kernel and the label is added

snapd rewrites the os_prefix= line of config.txt to the chosen kernel's folder and writes cmdline.txt there, adding snapd_recovery_mode and, outside run mode, snapd_recovery_system. A gadget refresh that replaces config.txt is followed by the same rewrite.

How do you point a bootloader that won't take instructions at a kernel? You rewrite one line of its configuration. The gadget's config.txt names the kernel, the command line file and the initrd as plain file names, with an empty os_prefix line. The firmware puts that prefix in front of every file it loads. When snapd changes the mode or the kernel_status, it rewrites that line with the chosen kernel's folder. Then it writes a command line file there: the gadget's first line, plus the mode. In recover or install mode, the prefix points at the recovery kernel and the label's added.

04 A one-time boot through tryboot.txt

TRYBOOT piboot.confsnap_try_kernel=pi-kernel_830.snapkernel_status=try tryboot.txtos_prefix=/piboot/ubuntu/pi-kernel_830.snap/config.txtos_prefix=/piboot/ubuntu/pi-kernel_812.snap/untouched: still the old kernel pi-kernel_830.snap/cmdline.txt... snapd_recovery_mode=run kernel_status=trying snapd: reboot/run/systemd/reboot-param0 trybootpassed to the reboot system call Pi firmware: uses tryboot.txtfor this boot only; a boot without the flag reads config.txt
With kernel_status=try snapd writes tryboot.txt pointing at pi-kernel_830.snap while config.txt still points at the old kernel, writes kernel_status=trying into the new kernel's cmdline.txt, and reboots with the argument 0 tryboot, so the firmware uses tryboot.txt for one boot

With kernel_status=try snapd writes tryboot.txt pointing at the new kernel and a cmdline.txt containing kernel_status=trying, then reboots with the argument 0 tryboot. config.txt is not touched. What the firmware does with the flag is firmware behaviour, outside snapd.

Your Pi has pi-kernel 830 unpacked. The firmware can't be told "try this once" in a variable, but it can be told in the reboot itself. snapd sets the try kernel and the status try. Because the status changed, piboot writes tryboot.txt this time. config.txt is left alone, still naming the old kernel. The try kernel's command line gets one extra argument: kernel_status=trying. When snapd asks for the reboot, piboot supplies an argument, 0 tryboot, which ends up in the kernel's reboot call. The firmware sees that flag and uses tryboot.txt for one boot.

05 The initramfs flips the status

THE FLIP piboot.conf sayskernel command line saysnew value in piboot.conf snap-bootstrapearly in a run boot, for a bootloader that cannot script kernel_status=try... kernel_status=trying kernel_status=trying kernel_status=tryno kernel_status=tryingkernel_status=reset to empty kernel_status=anythingunchanged: nothing to do this write saves piboot.conf only; config.txt and tryboot.txt are not regenerated
snap-bootstrap compares kernel_status in piboot.conf with kernel_status on the kernel command line: try and trying become trying; try without the argument is reset to empty; an empty status is left alone; the write saves only piboot.conf

snap-bootstrap sets kernel_status=trying only when piboot.conf says try and the kernel command line carries kernel_status=trying; any other non-empty combination is reset to empty. The write saves piboot.conf without regenerating config.txt or tryboot.txt.

The new kernel is running, but nobody has changed try to trying yet. On the Pi that job falls to the first program that can write a file. Early in a run boot, snap-bootstrap sees a bootloader that can't script. It reads the status from piboot's file and looks for kernel_status on the kernel command line. File says try, command line says trying? Then the firmware really did use tryboot, and the status becomes trying. Any other combination means the try didn't happen, so the status goes back to empty. An empty status means nothing to do. And this write only saves the file; nothing else is regenerated.

06 Success, failure and a power cut

OUTCOMES successsnap_try_kernel cleared,status emptytryboot.txt removedconfig.txt: os_prefixpi-kernel_830.snap new kernelfailsrestart withoutthe tryboot flagconfig.txt untouched:old kernel bootsno kernel_status=trying:status reset power cutbefore the rebootstatus is try, but a coldboot has no flagconfig.txt untouched:old kernel bootsno kernel_status=trying:status reset "we always have a valid config.txt that will allow booting"after a failure snapd sees the old kernel running and reverts the refresh
Three outcomes: on success snapd clears the try kernel, removes tryboot.txt and rewrites config.txt for pi-kernel_830.snap; if the new kernel fails, or the power is cut before the reboot, the firmware boots the old kernel from the unchanged config.txt and the initramfs resets the status

After a successful try snapd removes tryboot.txt and rewrites config.txt for the new kernel. A failed try, or a power cut before the reboot, boots the unchanged config.txt and the initramfs resets the status.

Three things can happen after that reboot, and only one of them changes config.txt. If the new kernel boots well, snapd makes it current and empties the status. piboot removes tryboot.txt and rewrites config.txt with the new prefix. If the new kernel fails, the next boot has no tryboot flag. The firmware reads the untouched config, the old kernel starts without trying on its command line, and the initramfs resets the status. A power cut after snapd wrote try, but before the reboot, ends the same way. Then snapd sees the old kernel and reverts.

07 The EEPROM has to know tryboot

EEPROM Raspberry Pi 4firmware older than 2021? recognised by the revision code: processor field 3checked before a kernel is extracted/proc/device-tree/chosen/bootloader/build-timestamp>= 16160576512021-03-18 yes: extract the kernel no: extraction fails"your EEPROM does not support tryboot, please upgrade to a newerone before installing Ubuntu Core" not checked when an image is being prepared, nor on other Pi models also: no managed kernel command line, no boot chain for sealing
On a Raspberry Pi 4, recognised by its revision code, snapd reads the EEPROM build timestamp from the device tree before extracting a kernel; at or after 1616057651 (2021-03-18) the kernel is extracted, otherwise extraction fails with an error asking to upgrade the EEPROM

On a Raspberry Pi 4 kernel extraction fails unless the EEPROM build timestamp is at least 1616057651 (2021-03-18): "your EEPROM does not support tryboot, please upgrade to a newer one before installing Ubuntu Core". piboot has no managed command line and no boot chain for sealing.

There's one way this can fail before it starts: a Pi 4 that's been sitting in a drawer since 2020. All of this needs firmware that understands tryboot. snapd recognises a Pi 4 from its revision code, and before unpacking a kernel it reads the bootloader's build timestamp from the device tree. That has to be the eighteenth of March 2021 or later. If it isn't, extraction fails, with a message telling you to upgrade the EEPROM before installing Ubuntu Core. The check is skipped while an image is built, and on other boards. Two more limits: no managed kernel command line, and no boot chain to seal keys to.

08 Where to go from here

ROADS Raspberry Pi: piboot and trybootTry, trying, donewhat the statuses mean Bootloadersone interface, compared U-Bootthe Pi before Ubuntu Core 22 Reboots and snap rebootfrom refresh to reboot The kernel command linenot managed on a Pi Ubuntu Coreback to the overview
Roads from piboot: try, trying and done; the bootloader interface; U-Boot; reboots; the kernel command line; and the Ubuntu Core overview

Roads: try and rollback, the bootloader interface, U-Boot, reboots, the command line and the overview.

That's a kernel try on a board whose firmware can't help. To compare it with boards that can, start with try, trying and done, which says what those statuses mean everywhere. The bootloaders deck puts piboot next to the other implementations. U-Boot is how the Pi booted before Ubuntu Core 22. The reboots deck shows how a refresh gets to the reboot that carries 0 tryboot. The command line deck covers the part piboot leaves out. And the Ubuntu Core overview takes you back to the start.

Read more

Files on ubuntu-seed

PathWritten byPurpose
piboot/ubuntu/piboot.confsnapdboot variables, U-Boot environment format, 4096 bytes
config.txtgadget, then snapdfirmware configuration; snapd replaces the os_prefix= line
tryboot.txtsnapda copy of config.txt for one try boot; removed afterwards
cmdline.txtgadgetthe base command line; snapd uses its first line
piboot/ubuntu/<snap-file>/snapdkernel.img, initrd.img, device trees, overlays/, cmdline.txt for a run kernel
systems/<label>/kernel/snapdthe same for a recovery system
start*, fixup*, bootcode*gadgetfirmware files

On a running system piboot/ubuntu is also mounted at /boot/piboot. The gadget marker is an empty piboot.conf in the gadget snap. The gadget's ubuntu-boot partition has no content.

Variables in piboot.conf

snapd_recovery_system, snapd_recovery_mode, snapd_recovery_kernel, kernel_status, snap_kernel, snap_try_kernel, snapd_extra_cmdline_args, snapd_full_cmdline_args, recovery_system_status, try_recovery_system.

What snapd generates

StateConfig file writtenos_prefixAdded to cmdline.txt
run modeconfig.txt/piboot/ubuntu/<snap_kernel>/snapd_recovery_mode=run
run mode, kernel_status=trytryboot.txt/piboot/ubuntu/<snap_try_kernel>/snapd_recovery_mode=run kernel_status=trying
any other modeconfig.txt/systems/<snapd_recovery_system>/kernel/snapd_recovery_mode=<mode> snapd_recovery_system=<label>

Files are regenerated when snapd_recovery_mode or kernel_status changes, and after a gadget refresh that replaced config.txt. Extra os_prefix= lines are commented out. Clearing snap_try_kernel removes tryboot.txt. While kernel_status is try, the reboot argument is 0 tryboot.

The status flip in the initramfs

kernel_status in piboot.confkernel_status on the command lineNew value
emptyanythingunchanged
trytryingtrying
anything elseanything elseempty

The new value is saved without regenerating config.txt or tryboot.txt.

Limits

  • Ubuntu Core 20 and later only; the pi gadget uses piboot from its 22 branch onwards.
  • Raspberry Pi 4: EEPROM build timestamp at least 1616057651 (2021-03-18). The check is made when a run kernel is extracted, not a recovery kernel, and is skipped when preparing an image and on other boards.
  • If the reboot has to be retried, the retry carries no 0 tryboot, so a Pi kernel refresh rolls back (daemon/daemon.go:663-672).
  • No trusted assets, no managed kernel command line, no boot chain for sealing.
  • No boot menu: the mode is whatever config.txt points at.

Further reading

The published documentation has no page on piboot.conf, tryboot.txt or os_prefix.

Roads from here

RUNNING SNAPS

  • Configuration — system options which on a Pi include settings written to config.txt

UBUNTU CORE · boot

UBUNTU CORE · bootloaders