Asking a .run script about its system
memory and users for CI operators for users
X.run --query prints the machine's memory in MB and each SSH user and password as group.key=value lines and exits without starting QEMU, so allocate and the CI components can check memory and log in without knowing the distribution.
On the map: runtime
▶ Watch this deck — 7 scenes, ~5 min
Worth seeing first
- The .run script — the script that answers
01 Ask the script, not a table
./ubuntu-cloud-24.04.x86_64.run --query prints the machine's memory and its users, one key=value per line, and exits before QEMU starts.
Say your CI job starts a different system on every run: Ubuntu today, Fedora tomorrow. To log in, it needs a user name and a password, and to start the machine safely, it should know how much memory the machine takes. You could keep a table, the way spread.yaml lists ubuntu and ubuntu by hand for each system. But the .run script already knows. Run it with --query, and instead of booting, it prints what it knows, one fact per line, and exits. QEMU doesn't start at all. This deck reads that answer for Ubuntu 24.04 on x86, and shows who relies on it.
Details and sources
Details
--queryis checked first thing in every launcher: when it is the first argument, the script prints its properties and runsexit 0, before any QEMU option is computed (010-qemu.mk:57-90). There is noimage-garden querycommand: the mode belongs to each.runscript.- The repository's own
spread.yamlgives each system'susernameandpasswordby hand (spread.yaml:60-72). - The property list and the format are documented in the QUERY file (QUERY:6-15).
- The launcher must exist:
image-garden make X.runmakes it, and with the defaultGARDEN_RUN_REQUIRES_QCOW2=1that makes the disk first (010-qemu.mk:14).
Read more
02 The answer, line by line
Most systems list root/root as user 0 and the distribution's user as user 1. ubuntu-core-* lists only ubuntu/ubuntu, as user 0.
Here's the whole answer for our example. The first line is the memory, in megabytes: two thousand and forty-eight. Then come the users, numbered from zero. User zero is root, with the password root. User one is the distribution's own user, ubuntu here, and its password is the same word. Every line is a dotted key, an equals sign and a value, so a tool can pick out the one it wants. Ubuntu Core is the system that differs. You can't log in as root there, so it lists a single user, ubuntu, as user zero, and there's no user one.
Details and sources
Details
- Run: both launchers were generated from the
image-garden.mkof the commit in garden.commit and run with--query; the output is the one on the slide, and matches the QUERY file's example (QUERY:55-69). - The users come from a
caseon a user mode:only:ubuntuprints user 0ubuntu/ubuntu;root distroprints user 0root/rootand user 1 named after the group, with its name as the password (010-qemu.mk:79-88). - rulegen.py's default mode is
root distroand it passes the group as the user name (rulegen.py:127, rulegen.py:596); the Ubuntu Core rules passonly:ubuntu(050-ubuntu-core.mk:25). - Indexes are zero-based; the format is
group.key=value(QUERY:8-9, QUERY:31-53). - That the guest really has these users and passwords is cloud-init's work: see users and passwords.
Read more
03 The memory follows your option
-m 2048, -m 512M and -m 4G give 2048, 512 and 4096. Other forms give vm.memory_mb=error: unsupported format … with exit status 0.
The memory line isn't a fixed number. It reads the same variable the script reads when it boots, QEMU_MEM_OPTION, and falls back to the same default, -m 2048. So when you export QEMU_MEM_OPTION as -m 4G, the answer says four thousand and ninety-six, which is what QEMU will get. It understands three forms: a plain number of megabytes, a number with M, and a number with G, in either case. Anything else gives a value that starts with the word error, and the script still exits with success. So check the value, not the exit code.
Details and sources
Details
- The query reads
${QEMU_MEM_OPTION=-m 2048}: the default applies only when the variable is unset; set to an empty string, it gives the error line (010-qemu.mk:66, default from 010-qemu.mk:9-12). - Accepted forms:
-m N,-m NMor-m Nm,-m NGor-m Ng(times 1024); anything else printsvm.memory_mb=error: unsupported format "…"and exits 0 (010-qemu.mk:67-77, QUERY:19-29). - Run:
QEMU_MEM_OPTION="-m 4G"gavevm.memory_mb=4096,-m 4096Mgave 4096, and-m size=4Ggavevm.memory_mb=error: unsupported format "-m size=4G"with exit status 0, followed by no user lines. - QEMU itself accepts more forms, such as
-m size=4G(derived from QEMU's option syntax); the query does not.
Read more
04 Who asks
allocate reads vm.memory_mb for its memory check. snapcraft-vm, rockcraft-vm and charmcraft-vm log in as system.users.0, with sudo -H unless it is root.
Two tools on this map ask. image-garden allocate asks before it starts a machine. It reads the memory line and compares it with what your host has free, minus a reserve, divided by one and a half for QEMU's own needs. If that's too little, it warns you and waits thirty seconds. The craft components for GitLab CI ask right after allocate has answered with an address. They take user zero and its password, and log in over SSH. On a cloud image that's root. For any other user, they put sudo in front of every command. Neither tool knows anything about the distribution.
Details and sources
Details
- allocate's
memory_checkruns./X.run --query 2>/dev/null, keepsvm.memory_mb, skips the check when the value is empty or an error, and warns when it exceeds(MemAvailable − reserve) / 1.5; the reserve isGARDEN_MEMORY_RESERVE, else 2048 MB in an x11 or wayland session and 1024 MB otherwise. The warning waits 30 seconds (image-garden:711-782). It runs once the launcher exists (image-garden:1066-1068). - snapcraft-vm reads
system.users.0.usernameand.passwordright afterallocate --spread, stops with "cannot log into SYSTEM: its image names no user" when there is none, and usessudo -Hwhen the user is not root (snapcraft-vm:372-381); rockcraft-vm and charmcraft-vm do the same (rockcraft-vm:356-363, charmcraft-vm:397-404). - They log in with
sshpass -e ssh … -p PORT USER@localhost(snapcraft-vm:231-242). - The components pick Ubuntu cloud systems, whose user 0 is root.
Read more
05 Reading it from your own tool
Read stdout only (the script runs with set -x), find lines by key, test for error:, and log in as system.users.0: ubuntu on Ubuntu Core, root elsewhere.
If you write such a tool yourself, four details matter. The script runs bash with tracing on, so every command it runs is echoed to standard error. Read standard output only, the way allocate throws standard error away. Look lines up by their key, not by their position: a comment in the script promises more keys. Check the memory value for the word error. And log in as user zero, without counting on a second user. Ubuntu Core's scripts list ubuntu alone. Every other system, named instances included, lists root first, then the distribution's own user.
Details and sources
Details
- Every launcher starts with
set -xeu(010-qemu.mk:59), so bash traces each command of the query to standard error; allocate reads it with2>/dev/null(image-garden:720). - The script's comment: "TODO: add more properties like vm.cpus, vm.arch, etc." (010-qemu.mk:65).
- allocate treats a value that starts with
error:as no answer (image-garden:722-725). - The users come from a
caseon the launcher's user mode:only:ubuntuprintsubuntuas user 0 alone;root distroprintsrootas user 0 and the group's user as user 1 (010-qemu.mk:80-88). Ubuntu Core's launchers passonly:ubuntu(050-ubuntu-core.mk:25); rulegen.py's and the named instances' passroot distro(020-instances.mk:89, also :110 and :131). Run:ubuntu-cloud-24.04@webserver.x86_64.run --queryprinted the same four user lines asubuntu-cloud-24.04.x86_64.run --query. QUERY says every.runscript exposes the users (QUERY:31-53).
Read more
06 One preamble for every launcher for contributors
EMIT_QEMU_RUN_SCRIPT_QUERY_PREAMBLE(launcher, download dir, root distro or only:ubuntu, group). rulegen.py and the instance rules pass root distro, Ubuntu Core only:ubuntu.
Every launcher gets its answer from one make definition, the query preamble, written at the top of the script. It takes four arguments: the launcher to write, the download directory for a warning comment, a user mode, and the distribution's user name, which is the group. The mode is root distro for ordinary systems, rulegen's default, and a mode that lists only ubuntu for Ubuntu Core. The preamble turns it into a case statement. The rules for named instances pass root distro as well, so an instance lists the same users as its base. CI runs a test script, check-query, on each launcher it builds.
Details and sources
Details
EMIT_QEMU_RUN_SCRIPT_QUERY_PREAMBLEwrites the#!/bin/bashline,set -xeu, a warning comment naming the download directory and the--queryblock (010-qemu.mk:57-92).- Callers: rulegen.py's launcher rules with
{img.query_users}, defaultroot distro(rulegen.py:127, rulegen.py:596, and the other architectures' rules); the Ubuntu Core rules withonly:ubuntu(050-ubuntu-core.mk:25, :32, :63); the instance rules withroot distroas well (020-instances.mk:89, :110, :131). - The
caseword is"$(3)", and the arm isroot\ distro): an escaped space in a pattern matches the plainroot distrothe callers pass (010-qemu.mk:80-88). Run: an instance's launcher printed root and the group's user, as its base's did. tests/check-query.shchecks thatvm.memory_mbis numeric, that each password equals its user, root and the group's user for ordinary systems, and onlyubuntufor Ubuntu Core (check-query.sh:1-111). CI runs it on each launcher it builds (.gitlab-ci.yml:592-593, run-systems.sh:64); named instances are not among them.
Read more
07 Where to go from here
Next: allocate's memory check, the craft components' login, the users in the guest, and named instances.
That's the query: a few lines a script prints about itself, the memory and the users, so a tool can start and log in to any system without a table of its own. From here, see how allocate uses the memory line before it starts a machine, and how the craft components log in as user zero. The users deck shows where those accounts come from inside the guest. And the instances deck explains named instances, whose scripts answer with the users of the system they were made from.
Details and sources
Read more
Basis of the statements on this page
Read in the image-garden source at the commit in garden.commit ("code"). The outputs were run: the launchers of ubuntu-cloud-24.04.x86_64 and of an instance ubuntu-cloud-24.04@webserver.x86_64 were generated from that commit's image-garden.mk and run with --query. Nothing was booted.
Query properties
| Key | Value | Systems |
|---|---|---|
vm.memory_mb | memory in MB from QEMU_MEM_OPTION (default -m 2048), or error: unsupported format "…" | every launcher |
system.users.0.username, .password | root, root | systems made by rulegen.py, and their named instances |
system.users.1.username, .password | the group, such as ubuntu or fedora, twice | systems made by rulegen.py, and their named instances |
system.users.0.username, .password | ubuntu, ubuntu | ubuntu-core-* |
Sources: 010-qemu.mk:57-92, QUERY.
Memory forms
QEMU_MEM_OPTION | vm.memory_mb |
|---|---|
| unset | 2048 |
-m 2048 | 2048 |
-m 512M, -m 512m | 512 |
-m 4G, -m 4g | 4096 |
empty, -m size=4G, anything else | error: unsupported format "…", exit 0 |
Reading it from a shell script
query=$(./ubuntu-cloud-24.04.x86_64.run --query 2>/dev/null) mem=$(sed -n 's/^vm\.memory_mb=//p' <<<"$query") user=$(sed -n 's/^system\.users\.0\.username=//p' <<<"$query") password=$(sed -n 's/^system\.users\.0\.password=//p' <<<"$query") case "$mem" in error:*|'') echo "cannot tell the memory size" >&2 ;; esac
The lookups follow snapcraft-vm (snapcraft-vm:374-376); the memory test follows allocate (image-garden:720-725).
Who reads it
| Reader | Keys | Use |
|---|---|---|
image-garden allocate | vm.memory_mb | memory check before starting |
| snapcraft-vm, rockcraft-vm, charmcraft-vm | system.users.0.* | SSH login, sudo -H unless root |
tests/check-query.sh | all | image-garden's CI |
Roads from here
RUNTIME · make rules and the CLI
- image-garden allocate — the memory check that reads vm.memory_mb before a machine starts
- Named instances — named instances, whose scripts list the users of their base system
- The .run script — the rest of the script, below the query
BUILT ON IT · CI integrations
- Building snaps, rocks and charms in a VM — a CI job that logs in as user zero
- What the integrations rely on — the other image-garden behaviour the CI components rely on
FIRST BOOT · cloud-init changes
- The users every system gets — where the users and passwords come from inside the guest