Provisioning runs once
power off, cloud-init off for everyone
The provisioning boot ends with cloud-init powering the VM off, which ends make's recipe, and its last runcmd command creates /etc/cloud/cloud-init.disabled, so every later boot, from the .run script or allocate, starts as prepared without provisioning again.
On the map: first boot
▶ Watch this deck — 7 scenes, ~5 min
Worth seeing first
- From a system's name to a running VM — the overlay, the seed and the one boot this deck looks into
01 How does make know it's done?
make waits for QEMU to exit. The guest ends the provisioning boot by powering off, and switches cloud-init off for every later boot.
Making the image means booting it once, and make has to wait for that boot to finish. But how would it know cloud-init is done? It can't look inside the guest. The answer is simple: the guest switches itself off. When it powers off, QEMU exits, the recipe ends, and make moves on to the .run script. That leaves a second question. The image will boot many more times, from the run script and from allocate, and those boots must not set it up again. This deck shows both halves: the power-off at the end, and the switch that keeps cloud-init quiet afterwards.
Details and sources
Details
- The image rule creates the overlay, then runs QEMU in the foreground with the seed attached, piped through
tee X.qcow2.log; the recipe ends when QEMU exits (rulegen.py:601-609). The.runrule is ordered after the image, through| X.qcow2whenGARDEN_RUN_REQUIRES_QCOW2is set, which it is by default (rulegen.py:594, 010-qemu.mk:14). - The two halves are the
power_stateblock (010-qemu.mk:190-195) andtouch /etc/cloud/cloud-init.disabled(010-qemu.mk:246). - Derived: QEMU exits when the guest powers off because the command line has no
-no-shutdown.
Read more
02 Power off at the end
power_state: {delay: now, mode: poweroff, message: Powering off, timeout: 0, condition: true}. The console goes to ubuntu-cloud-24.04.x86_64.qcow2.log.
The first half is a power state block near the top of the user-data. It asks for a power-off, with no delay, the message Powering off, no timeout, and a condition that's simply true. cloud-init acts on it at the very end, after every command in the runcmd list has run. The guest shuts down, QEMU exits, and make's recipe is done. Everything the console printed along the way went through tee into a log next to the image, so for ubuntu-cloud-24.04.x86_64 that's its .qcow2.log. It's the first place to look when a provisioning boot goes wrong.
Details and sources
Details
- The block in the common template:
power_state:withdelay: now,mode: poweroff,message: Powering off,timeout: 0,condition: true(010-qemu.mk:190-195). The Ubuntu Core template has the same block (050-ubuntu-core.mk:135-140). - The provisioning command is
set -o pipefail && <QEMU command> … 2>&1 | tee X.qcow2.log(rulegen.py:606-609); the display defaults to-nographic, so the serial console reaches the log (010-qemu.mk:13). - Derived, from cloud-init's behaviour: the power-state module runs at the end of cloud-init's final stage, after the
runcmdscript, andcondition: truemeans it acts whatever happened before. - Amazon Linux 2 does not act on
power_state, and its template powers off by hand: see what differs per distribution.
Read more
03 Then cloud-init is switched off
The last runcmd command is touch /etc/cloud/cloud-init.disabled. It stops cloud-init on every later boot, not on this one.
The second half is the last command in the runcmd list. It creates an empty file, cloud-init.disabled, in cloud-init's config directory. When cloud-init starts and finds that file, it does nothing at all. The file is written during the provisioning boot, but it only matters on the next one, so this boot still finishes its list and powers off. From then on, the image boots as prepared: the same users, the same host name, the same SSH settings, and no second round of setup.
Details and sources
Details
touch /etc/cloud/cloud-init.disabledis the last command of the common template'sruncmdlist, after pkgproxy's late setup (010-qemu.mk:238-247).- Derived, from cloud-init's behaviour: at boot, cloud-init's systemd generator looks for
/etc/cloud/cloud-init.disabledand, when it exists, keeps cloud-init's services from running. A file created while cloud-init runs does not stop that run. - The Ubuntu Core template has no such command (050-ubuntu-core.mk:131-177); see Ubuntu Core images.
- Commands that a project template adds after
$(CLOUD_INIT_USER_DATA_TEMPLATE)run after this one, in the same boot; see how a user-data template is chosen.
04 Every later boot
.run keeps changes in the overlay; allocate adds -snapshot and keeps none. Neither provisions again; a new user-data makes make prepare the image again.
So what happens on the boots that follow? The .run script boots the prepared overlay with no seed attached, cloud-init stays off, and you get a login prompt in seconds. Changes you make there are written to the overlay, and they stay. allocate starts the same script with QEMU's snapshot option, so each test machine begins in the prepared state, and its changes are thrown away when it's discarded. Either way, provisioning doesn't repeat. It happens again only when make prepares the image again, for example after you edit your .image-garden.mk, which changes the user-data.
Details and sources
Details
- The
.runscript attaches only the overlay, as the boot disk (010-qemu-x86_64.mk:59-108); writes go toX.qcow2, the base image stays as downloaded. allocateruns the script with-snapshotunlessQEMU_SNAPSHOT_OPTIONsays otherwise, in the background with a pid file (image-garden:1153-1163). Derived, from QEMU's behaviour: with-snapshotwrites go to a temporary file and are lost when QEMU exits.- The user-data depends on the make files and
.image-garden.mk(rulegen.py:467-471); the image depends on the seed made from it (rulegen.py:601-604), so a new user-data means a new overlay and a new provisioning boot. - "In seconds" depends on the host and on KVM; under TCG emulation a boot takes longer.
Read more
05 What runs when in the provisioning boot for contributors
bootcmd, then write_files and packages, then runcmd in order, with template lines after the touch, then power_state.
When you write a template, the order inside this boot matters. bootcmd runs first, early on: that's where pkgproxy's hosts line and apt setup go, so that cloud-init's own package installs already use it. Then files are written, and any packages your template lists are installed. Then the runcmd list runs top to bottom: hosts, SSH, apt, pkgproxy's late setup, and the touch that disables cloud-init. Lines your template adds after the common one come after that touch, which is fine, because the file only stops later boots. Last of all, the power state switches the machine off.
Details and sources
Details
- The comment above pkgproxy's parts:
PKGPROXY_BOOTCMD_SETUP"runs from bootcmd, early in the first boot, so that cloud-init's own package installs (packages: in a custom template) already go through pkgproxy", andPKGPROXY_RUNCMD_SETUP"runs from runcmd, at the end of the first boot", because the snap script restarts snapd (010-qemu.mk:108-126). The two blocks are at 010-qemu.mk:156-168. - The common
runcmdlist: hosts, SSH snippet,PermitRootLoginsed, apt sed, pkgproxy's late setup, thentouch /etc/cloud/cloud-init.disabled(010-qemu.mk:238-246). - A template that starts with
$(CLOUD_INIT_USER_DATA_TEMPLATE)and adds dash lines extends that list after thetouch; Amazon Linux 2'sshutdown --poweroff nowis such a line (050-amazonlinux-2.mk:4-8). - Derived, from cloud-init's behaviour:
bootcmdruns in an early stage on every boot cloud-init runs;write_filesruns before the package module; the package module runs before theruncmdscript; the power-state module runs last.
06 When the boot doesn't end, or ends badly
No timeout: a guest that never powers off keeps make waiting. A failed command still powers off and counts as success. An interrupted or failed boot deletes X.qcow2.
What if the boot never ends? make sets no time limit on it. If the guest never powers off, because a template left out the common one or the system ignores the power state, make simply waits. Watch the console log, which tee writes as it goes. A command that fails doesn't stop the power-off either. The condition is true, so the machine shuts down anyway, QEMU exits cleanly, and make calls the image done. So read the log for errors. And if you interrupt make, or QEMU itself fails, make deletes the half-made image, and the next run starts the provisioning boot over.
Details and sources
Details
- The provisioning command has no timeout:
set -o pipefail && <QEMU command> … 2>&1 | tee X.qcow2.log(rulegen.py:606-609). Withpipefail, a QEMU that exits non-zero fails the recipe even thoughteesucceeds. .DELETE_ON_ERROR:removes a target whose recipe failed (000-garden.mk:4-7); The pattern.PRECIOUS: %.qcow2 %.img(010-qemu.mk:334) is meant for the converted downloads; it does not protect the overlay, whose rule is explicit. Run for this map with GNU make 4.4.1 on a two-line makefile of the same shape: a failing explicitx.qcow2rule had its target deleted. On an interrupt GNU make deletes a target that is not precious (derived).condition: trueinpower_state(010-qemu.mk:190-195). Derived, from cloud-init's behaviour: a failingruncmdcommand does not stop the rest of the list or the power-off, so QEMU exits with status 0.- Leaving out the common template removes
power_state: see start from the common template. Amazon Linux 2 ignorespower_stateand has its ownshutdown: see what differs per distribution. - The Ubuntu Core rules run the same way, under bash with
set -o pipefail(050-ubuntu-core.mk:37-53, 050-ubuntu-core.mk:77-106), so a QEMU that fails there fails the build too.
Read more
07 Where to go from here
Next: the seed, templates and hooks, allocate, the per-distribution quirks, and the serial console.
One boot that powers itself off, and a file that keeps cloud-init quiet from then on. The seed deck shows what that boot reads and how it gets there. To add your own setup to it, start with how a template is chosen, or with the simpler hooks. allocate is where the prepared image earns its keep, a fresh snapshot for every test. The quirks deck has the system that can't power itself off. And the console deck helps when you're reading that log.
Details and sources
Read more
Basis of the statements on this page
Read in rulegen.py, mk/000-garden.mk, mk/010-qemu.mk and image-garden of image-garden at the commit this map describes ("code"). When cloud-init runs each part is cloud-init's behaviour ("derived"). One run: a GNU make 4.4.1 test that a failing explicit %.qcow2 target is deleted under .DELETE_ON_ERROR despite the pattern .PRECIOUS.
The provisioning boot, in order
| Step | From | Notes |
|---|---|---|
bootcmd | pkgproxy's part (010-qemu.mk:156-162) | only when a pkgproxy is reachable from guests |
write_files, users, packages | the template | packages: only in a project template |
runcmd, the common part | 010-qemu.mk:238-246 | ends with touch /etc/cloud/cloud-init.disabled |
runcmd, a template's lines | the project or a quirk | after the touch; Amazon Linux 2's shutdown --poweroff now |
power_state | 010-qemu.mk:190-195 | poweroff, delay: now, condition: true |
After the boot
| Seed | cloud-init | Changes kept | |
|---|---|---|---|
provisioning boot (make X.qcow2) | attached | runs once | in X.qcow2 |
./X.run | not attached | off | in X.qcow2 |
image-garden allocate | not attached | off | no: -snapshot |
When it goes wrong
| What happens | What make does | What to do |
|---|---|---|
| the guest never powers off | waits; there is no timeout | read X.qcow2.log; check that the template starts from the common one |
a runcmd command fails | finishes; the image counts as made (derived) | read X.qcow2.log; rm X.qcow2 and make again |
| QEMU exits non-zero | fails the recipe (set -o pipefail) and deletes X.qcow2 (000-garden.mk:4-7) | fix the cause and make again |
| make is interrupted | deletes X.qcow2 (derived) | make again |
Roads from here
RUNTIME · make rules and the CLI
- image-garden allocate — how each test machine starts from the prepared image
- Working on the serial console — reading the console and its log
FIRST BOOT · cloud-init changes
- The NoCloud seed — what the provisioning boot reads and how it is attached
- How a user-data template is chosen — how to add your own setup to that boot
- Changing what cloud-init does — the simpler hooks for files and apt settings
- What differs per distribution — Amazon Linux 2, which powers off by hand