What pkgproxy adds to user-data
hosts, apt, snap, GOPROXY for CI operators
With a pkgproxy in reach when make writes a system's user-data, cloud-init adds pkgproxy to /etc/hosts, runs pkgproxy's apt setup script from bootcmd on Debian and Ubuntu and its snap setup script from runcmd on every family but Alpine, and appends GOPROXY to /etc/environment; Ubuntu Core gets none of it.
On the map: pkgproxy
▶ Watch this deck — 7 scenes, ~5 min
Worth seeing first
- How a guest reaches pkgproxy — what the name pkgproxy means in the guest
01 Three places in one file
With a pkgproxy, ubuntu-cloud-24.04.x86_64.user-data gains a bootcmd block, an /etc/environment entry in write_files and a runcmd item. The apt and snapd settings come from pkgproxy's own setup scripts.
Your guest's apt, snapd and Go all need to be pointed at pkgproxy, and it has to happen in the very first boot. So when make prepares ubuntu-cloud-24.04.x86_64 with a pkgproxy in reach, the user-data it writes grows in three places: a bootcmd block near the top, a file to append in write_files, and one more item in runcmd. Without a pkgproxy, those spots are empty and the rest is the same. What you won't find is an apt proxy file or a snapd setting written by image-garden. The guest fetches pkgproxy's own setup scripts instead, so the setup always matches the pkgproxy that serves it.
Details and sources
Details
- The common template puts
$(call PKGPROXY_CLOUD_INIT_BOOTCMD,$1)afterssh_pwauth,$(INJECTED_CLOUD_INIT_FILES)at the end ofwrite_filesand$(call PKGPROXY_CLOUD_INIT_RUNCMD,$1)before the lastruncmditem, which creates/etc/cloud/cloud-init.disabled(010-qemu.mk:187-247). Without a pkgproxy all three are empty (010-qemu.mk:181-185). - "Guests know pkgproxy as "pkgproxy" and set themselves up with the scripts it serves under /setup/, so that they always match the pkgproxy in use" (010-qemu.mk:108-109); "image-garden keeps no copies of them" (proxy/README.md:147-152).
tests/check-pkgproxy-user-data.pychecks that image-garden writes no apt or snapd configuration of its own. - Run for this deck:
image-garden make ubuntu-cloud-24.04.x86_64.user-datawith a fixture socket, and again with nothing set; the two files differ only in the three places. - The port is the URL's with
PKGPROXY_URL, and 3142 with the snap's sockets (010-qemu.mk:128). - A project template that does not start from the common template loses these lines: see start from the common template.
02 A name for the host
10.0.2.100 pkgproxy (or the URL's address) goes into /etc/hosts in bootcmd and again in runcmd, unless it is there. Scripts come from http://pkgproxy:PORT/setup/TOOL.
Every piece starts with a name. The first line of the bootcmd block adds pkgproxy to /etc/hosts, with the address make chose: 10.0.2.100 for the snap's socket, or the address of your URL's host. It adds the line only if it isn't there yet, and the runcmd item adds it again, in case cloud-init rewrote the hosts file in between. From then on, everything in the guest just says pkgproxy and the port. The setup scripts come from there too, from pkgproxy's setup path, fetched with curl, or with wget where there's no curl.
Details and sources
Details
PKGPROXY_HOSTS_SHELL: "A shell line that adds pkgproxy to /etc/hosts unless it is there. It runs from runcmd as well, in case cloud-init rewrote /etc/hosts since bootcmd" (010-qemu.mk:132-134). It is the first line of the bootcmd block and of the runcmd item (010-qemu.mk:156-168).- The lines on the slide are from
ubuntu-cloud-24.04.x86_64.user-datamade with a fixture socket; withPKGPROXY_URL=http://192.0.2.9:3142only the address changes. PKGPROXY_GUEST_URL = http://pkgproxy:$(PKGPROXY_PORT)(010-qemu.mk:128).PKGPROXY_SETUP_SHELLfetches each script withcurl -fsSL --connect-timeout 5 http://pkgproxy:PORT/setup/"$tool" | sh, orwget -q -T 5 -O - … | sh, or printsimage-garden: no curl or wget to fetch …(010-qemu.mk:150-154). "Only the connection has a short timeout: the scripts are small, but the network may be slow."- Every line of the blocks is POSIX sh without backslashes, "which the echo that writes the user-data could take as escapes" (010-qemu.mk:146-147).
- The base template's own
/root/hosts.APPEND, which maps the guest's host name to127.0.0.1, is appended in runcmd too, before the pkgproxy item (010-qemu.mk:231-245).
Read more
- How a guest reaches pkgproxy, where the address comes from.
03 apt, early, from bootcmd
On Debian and Ubuntu, bootcmd probes pkgproxy once, then runs /setup/apt, which writes Acquire::http::Proxy to /etc/apt/apt.conf.d/99pkgproxy before cloud-init installs packages.
On Debian and Ubuntu, apt is set up first, from bootcmd. That's very early in the first boot, before cloud-init installs any packages, and it matters when your template adds a packages list: those installs already go through pkgproxy. The block asks pkgproxy once whether it's there at all. Then it fetches the apt setup script and hands it to the shell. The script writes apt's HTTP proxy setting into a file under apt's configuration directory. apt keeps its usual mirrors and simply fetches through the proxy. On every other family this step is left out.
Details and sources
Details
PKGPROXY_BOOTCMD_SETUP = $(if $(filter debian ubuntu,$1),apt)(010-qemu.mk:129). It "runs from bootcmd, early in the first boot, so that cloud-init's own package installs (packages: in a custom template) already go through pkgproxy" (010-qemu.mk:114-116).- The bootcmd block is the hosts line, the probe and, where the group has one, the setup line (010-qemu.mk:156-162). The probe asks
http://pkgproxy:PORT/once and, without an answer, leaves/run/image-garden-pkgproxy-unreachable, so that neither batch waits again in that boot (010-qemu.mk:122-126): see an unreachable pkgproxy. - pkgproxy's apt script "writes
Acquire::http::Proxyto/etc/apt/apt.conf.d/99pkgproxy" (proxy/README.md:171-174). The script itself is pkgproxy's, served by the running daemon; its exact content is in pkgproxy's tree, not image-garden's. - apt is "a real forward proxy. Clients keep their real
sources.listmirror URLs and pointAcquire::http::Proxyat the proxy" (proxy/README.md:40-41). - Separately, the common template's runcmd rewrites
https://tohttp://in every*.listand deb822*.sourcesfile under/etc/apt, with or without a pkgproxy, so that what apt fetches is plain http that pkgproxy can cache (010-qemu.mk:244); see SSH logins and apt sources.
04 snapd, late, from runcmd
On every family but Alpine, runcmd runs /setup/snap: a drop-in for snapd.service with SNAPPY_FORCE_API_URL, written even before snapd is installed, and a restart of snapd where it is.
snapd is set up last, from runcmd, at the end of the first boot, on every family but Alpine. It can't go earlier. The snap setup script restarts snapd, the restart command waits until snapd is back, and snapd may be ordered after cloud-init's early stages, so from bootcmd that wait could hold up the boot. The script writes a systemd drop-in that points snapd at pkgproxy as if it were the snap store. It writes the drop-in even where snapd isn't installed yet, so a snapd your tests install later uses pkgproxy too. Right after this item, cloud-init is switched off.
Details and sources
Details
PKGPROXY_RUNCMD_SETUP = $(if $(filter alpine,$1),,snap)(010-qemu.mk:130). "PKGPROXY_RUNCMD_SETUP runs from runcmd, at the end of the first boot. The snap script restarts snapd, which must not happen from bootcmd: systemctl waits for the restart, and snapd may be ordered after cloud-init's early stages" (010-qemu.mk:117-120). That the wait would hold up the boot is the slide's reading of that comment (derived).- The runcmd item is the hosts line and the setup line (010-qemu.mk:164-168), placed just before
touch /etc/cloud/cloud-init.disabled(010-qemu.mk:245-246). - pkgproxy's snap script "writes a systemd drop-in for snapd (
/etc/systemd/system/snapd.service.d/pkgproxy.conf) that setsSNAPPY_FORCE_API_URL, also where snapd is installed later, and restarts snapd where it is installed" (proxy/README.md:175-180). snapd is "pointed at the proxy directly, as if it were the store … not a traditional proxy setting" (proxy/README.md:42-44). pkgproxy=noneexists mainly for snapd's own spread suite, where some tests are not yet adjusted for the injectedSNAPPY_FORCE_API_URLor the extra/etc/hostsentries (proxy/README.md:134-136).
05 Go, through /etc/environment
write_files appends GOPROXY="http://pkgproxy:3142/go,direct" to /etc/environment, which PAM reads for every SSH session, before and after Go is installed.
Go is the odd one out. The go command has no system-wide configuration file, and pkgproxy's go script sets up only the user who runs it, and only where Go is installed, which at first boot it usually isn't. So image-garden doesn't run that script. It appends one line to /etc/environment instead: GOPROXY, pointing at pkgproxy's go path, then ,direct, Go's way back to the module's own source. That file is read for every SSH session, interactive or not. So a shell you log into, and a one-off go build over SSH, both get the proxy.
Details and sources
Details
- "The go command has no system-wide configuration, and pkgproxy's setup script for it configures the user who runs it, where go is installed: at first boot it usually is not. /etc/environment reaches every user's SSH session, before and after go is installed" (010-qemu.mk:170-173). The
write_filesentry isINJECTED_CLOUD_INIT_FILES(010-qemu.mk:174-180). go env -w, which pkgproxy's script uses, "only writes the invoking user's own$GOENV";/etc/environmentis read by PAM (pam_env, enabled by the template'sUsePam yessshd setting) for every SSH session (proxy/README.md:192-202, 010-qemu.mk:212-220).- The README says the trailing
,directkeeps modules pkgproxy can't serve reachable from their source "onceGOPRIVATEexcludes them" (proxy/README.md:200-202). In Go'sGOPROXYlist a comma moves to the next entry only when the proxy answers 404 or 410 (derived: Go's documented behaviour). - The port is the URL's with
PKGPROXY_URL; 3142 with the snap's sockets. - The craft components run pkgproxy's go script themselves, as root, because snapcraft builds as root: see before the build, the setup scripts.
Read more
06 Family by family
Debian, Ubuntu: hosts, apt, snap, GOPROXY. Other families: hosts, snap, GOPROXY. Alpine: hosts, GOPROXY. Ubuntu Core: none. A project's .image-garden.mk may redefine PKGPROXY_BOOTCMD_SETUP and PKGPROXY_RUNCMD_SETUP.
Put together, each family gets its own mix. Debian and Ubuntu get all four: the hosts line, apt, snapd and GOPROXY. Fedora, Arch, openSUSE and the other cloud families get the hosts line, snapd and GOPROXY. Alpine gets the hosts line and GOPROXY. Ubuntu Core gets none of it, because its user-data comes from a template of its own. If your project needs a different mix, your .image-garden.mk can redefine the two variables that pick the scripts, PKGPROXY_BOOTCMD_SETUP and PKGPROXY_RUNCMD_SETUP. Each is called with the guest's group, and returns the names of the scripts to run.
Details and sources
Details
PKGPROXY_BOOTCMD_SETUP = $(if $(filter debian ubuntu,$1),apt)andPKGPROXY_RUNCMD_SETUP = $(if $(filter alpine,$1),,snap)(010-qemu.mk:129-130). The scripts are picked per family because "pkgproxy's setup scripts do nothing where their tool is missing, but there is no one script for a whole guest yet (zygoon/pkgproxy#92)" (010-qemu.mk:110-112).- The groups in
data/*.iniare almalinux, alpine, amazonlinux, archlinux, centos, debian, fedora, gentoo, opencloudos, opensuse, oracle, rocky and ubuntu; all but alpine, debian and ubuntu get the "other families" row. - "A
.image-garden.mkcan override either variable;$1is the guest's group, such asdebianorfedora" (proxy/README.md:182-184). The example on the slide is illustrative, not taken from the tree; an empty result leaves the setup line out of that block. - "Ubuntu Core guests are not set up for pkgproxy" (proxy/README.md:204): their user-data comes from
BASE_UBUNTU_CORE_CLOUD_INIT_USER_DATA_TEMPLATE, which has no pkgproxy lines (050-ubuntu-core.mk:131-179). - A release or group template that starts from the common template keeps these lines; one written from scratch does not: see your template replaces these.
tests/check-pkgproxy-config.shchecks one system of each of debian, ubuntu, fedora, alpine, amazonlinux and opensuse (check-pkgproxy-config.sh:24-25).
Read more
07 Where to go from here
Next: reaching pkgproxy, an unreachable pkgproxy, the common template, your .image-garden.mk and why made images keep their setup.
That's everything pkgproxy adds: a name in the hosts file, apt set up early, snapd set up late, and GOPROXY for every login. To see where the name's address comes from, take the reaching deck. If you wonder what the probe does when nothing answers, the unreachable deck has it. The template decks show the file these blocks live in, and how your own .image-garden.mk changes it. And the rebuild deck explains why images made before you change the setting keep what they had.
Details and sources
Read more
Basis of the statements on this page
Read in mk/010-qemu.mk, mk/050-ubuntu-core.mk and proxy/README.md of image-garden at the v0.8.0 tag ("code"). The excerpts below were produced by running image-garden make ubuntu-cloud-24.04.x86_64.user-data from an export of that tag, once with a fixture $SNAP_COMMON holding pkgproxy.sock and once with nothing set ("run"). What pkgproxy's setup scripts write is from proxy/README.md: the scripts are served by pkgproxy and are not in image-garden's tree.
The three additions, as made
With the snap's socket (10.0.2.100, port 3142). With PKGPROXY_URL=http://192.0.2.9:3142, only the address on the two hosts lines changes.
bootcmd:
- |
grep -q '^10.0.2.100 pkgproxy$' /etc/hosts || { echo '# image-garden: pkgproxy, see http://pkgproxy:3142/'; echo '10.0.2.100 pkgproxy'; } >>/etc/hosts
if command -v curl >/dev/null 2>&1; then curl -s -o /dev/null --connect-timeout 5 --max-time 20 http://pkgproxy:3142/; elif command -v wget >/dev/null 2>&1; then wget -q -T 5 -t 1 -O /dev/null http://pkgproxy:3142/; fi || { echo "image-garden: cannot reach pkgproxy at http://pkgproxy:3142/, skipping its setup scripts in this boot" >&2; touch /run/image-garden-pkgproxy-unreachable; }
if [ -e /run/image-garden-pkgproxy-unreachable ]; then echo "image-garden: pkgproxy was unreachable, skipping setup scripts: apt" >&2; else for tool in apt; do if command -v curl >/dev/null 2>&1; then curl -fsSL --connect-timeout 5 http://pkgproxy:3142/setup/"$tool" | sh; elif command -v wget >/dev/null 2>&1; then wget -q -T 5 -O - http://pkgproxy:3142/setup/"$tool" | sh; else echo "image-garden: no curl or wget to fetch http://pkgproxy:3142/setup/$tool" >&2; fi; done; fi
write_files:
...
- path: /etc/environment
permissions: '0644'
append: true
content: |
GOPROXY="http://pkgproxy:3142/go,direct"
runcmd:
...
- |
grep -q '^10.0.2.100 pkgproxy$' /etc/hosts || { echo '# image-garden: pkgproxy, see http://pkgproxy:3142/'; echo '10.0.2.100 pkgproxy'; } >>/etc/hosts
if [ -e /run/image-garden-pkgproxy-unreachable ]; then echo "image-garden: pkgproxy was unreachable, skipping setup scripts: snap" >&2; else for tool in snap; do if command -v curl >/dev/null 2>&1; then curl -fsSL --connect-timeout 5 http://pkgproxy:3142/setup/"$tool" | sh; elif command -v wget >/dev/null 2>&1; then wget -q -T 5 -O - http://pkgproxy:3142/setup/"$tool" | sh; else echo "image-garden: no curl or wget to fetch http://pkgproxy:3142/setup/$tool" >&2; fi; done; fi
- touch /etc/cloud/cloud-init.disabledWhat each setup changes in the guest
| Piece | Runs from | Families | What it leaves in the guest |
|---|---|---|---|
| hosts line | bootcmd and runcmd | all but Ubuntu Core | ADDRESS pkgproxy and a comment in /etc/hosts |
| probe | bootcmd | all but Ubuntu Core | /run/image-garden-pkgproxy-unreachable when pkgproxy does not answer |
/setup/apt | bootcmd | debian, ubuntu | Acquire::http::Proxy in /etc/apt/apt.conf.d/99pkgproxy |
/setup/snap | runcmd | all but alpine and Ubuntu Core | /etc/systemd/system/snapd.service.d/pkgproxy.conf with SNAPPY_FORCE_API_URL; snapd restarted where installed |
GOPROXY | write_files | all but Ubuntu Core | GOPROXY="http://pkgproxy:PORT/go,direct" appended to /etc/environment |
Sources: 010-qemu.mk:127-185, proxy/README.md:166-204.
The make variables
| Variable | What it is | Override in .image-garden.mk |
|---|---|---|
PKGPROXY_GUEST_URL | http://pkgproxy:PORT | no need |
PKGPROXY_BOOTCMD_SETUP | scripts for bootcmd, called with the group as $1; apt for debian and ubuntu | yes |
PKGPROXY_RUNCMD_SETUP | scripts for runcmd, called with the group; snap for all but alpine | yes |
PKGPROXY_HOSTS_SHELL | the hosts line | not documented |
PKGPROXY_PROBE_SHELL | the probe | not documented |
PKGPROXY_UNREACHABLE_FILE | /run/image-garden-pkgproxy-unreachable | not documented |
PKGPROXY_SETUP_SHELL | the loop that fetches and runs scripts | not documented |
INJECTED_CLOUD_INIT_FILES | the /etc/environment entry | not documented |
All of them are defined only when make found a pkgproxy; otherwise the three blocks are empty (010-qemu.mk:181-185).
Roads from here
BUILT ON IT · CI integrations
- Why the craft components keep no cache — the craft components' own setup run in the VM
FIRST BOOT · cloud-init changes
- How a user-data template is chosen — the common user-data template these blocks sit in
- Changing what cloud-init does — changing the user-data from your project's .image-garden.mk
PKGPROXY · caching for guests
- How a guest reaches pkgproxy — where the address in the hosts line comes from
- An unreachable pkgproxy — what bootcmd does when pkgproxy does not answer
- Changing the proxy does not rebuild — why a changed setting leaves made images as they are
- What pkgproxy gives a garden — what pkgproxy gives a garden
pkgproxy map ↗
- Setting up clients — on the pkgproxy map, what the setup scripts do
- apt through a forward proxy — on the pkgproxy map, apt through a forward proxy
- Go module proxy — on the pkgproxy map, the Go module proxy