pkgproxy from a container

PKGPROXY_URL only for CI operators

The container has no pkgproxy and looks for no pkgproxy socket, so PKGPROXY_URL names one at an address that both the container and the guests inside it reach; GARDEN_DL_PKGPROXY sends image downloads elsewhere, and an unreachable pkgproxy is forgotten when the container exits.

On the map: container

▶ Watch this deck — 5 scenes, ~4 min

Worth seeing first

01 No pkgproxy inside, one variable in

ONE VARIABLE IN the container image-garden QEMU the guest ubuntu-cloud-24.04.x86_64 no pkgproxy inside no socket mode docker run -e PKGPROXY_URL=http://pkgproxy.lan:3142 ... pkgproxy on your network cloud images packages, snaps, Go modules
The container holds image-garden and QEMU with the guest ubuntu-cloud-24.04.x86_64, but no pkgproxy and no socket mode. docker run -e PKGPROXY_URL=http://pkgproxy.lan:3142 passes one address in: image-garden fetches cloud images through that pkgproxy on your network, and the guest sends its packages, snaps and Go modules there.

No pkgproxy in the image and no socket mode: -e PKGPROXY_URL=http://HOST:PORT is read by image-garden for image downloads and by make for the guests it prepares.

The container gives you QEMU, but not pkgproxy. Nothing in the image runs one, and its make looks for a pkgproxy socket only where the snap keeps it, so the socket bridge in its patched QEMU goes unused. That leaves one way in: the PKGPROXY_URL variable, passed with -e. Two programs read it. image-garden itself, in the container, asks that pkgproxy for every cloud image. And make writes it into each guest it prepares, for packages, snaps and Go modules. For ubuntu-cloud-24.04.x86_64, that's the image download, and then every apt install in the guest, through one proxy.

Details and sources

Details

  • The final stage installs no pkgproxy and starts no service; the image has no entrypoint (Dockerfile:73-103). The README's comparison gives the container "With PKGPROXY_URL" for pkgproxy (README.md:104).
  • make takes a URL first; otherwise it bridges a socket only when $(PKGPROXY_SOCKET_DIR)/pkgproxy.sock exists, and PKGPROXY_SOCKET_DIR is built from SNAP_COMMON (mk/010-qemu.mk:20-44). In the container SNAP_COMMON is unset, so make tests /pkgproxy.sock, which the image does not have (derived). The image's QEMU carries the guestfwd patch all the same (Dockerfile:29-32); see what you lose without it.
  • image-garden uses PKGPROXY_URL for image downloads unless GARDEN_DL_PKGPROXY is set (image-garden:31-38); make writes the guest's /etc/hosts entry, the apt and snap setup and GOPROXY into its user-data (mk/010-qemu.mk:127-180).
  • pkgproxy.lan on the slide is an example name. Outside the snap, make PKGPROXY_URL=… and PKGPROXY_URL in the environment work the same (proxy/README.md:121-127).

02 One address, two places to reach it from

TWO PLACES, ONE ADDRESS your machine the container pkgproxy on your network image-garden QEMU's user network the guest getent ahostsv4 HOST the name is resolved here 127.0.0.1: the container 127.0.0.1: the guest itself 10.0.2.2: still the container -e PKGPROXY_URL=http://pkgproxy.lan:3142 its address on your network
On your machine, the container holds image-garden and QEMU's user network with the guest inside it. image-garden connects to pkgproxy from the container's network; the guest goes out through QEMU and the container. The name is resolved inside the container. 127.0.0.1 is the container for image-garden and the guest for the guest, and 10.0.2.2 only reaches the container; PKGPROXY_URL should name the pkgproxy machine's address on your network.

image-garden connects from the container, the guest from behind QEMU's user network inside it. 127.0.0.1 and 10.0.2.2 stay inside the container; use the pkgproxy machine's network address.

So the address has to work from two places. image-garden connects from the container's network. The guest sits one level further in, behind QEMU's user network, and goes out through the container's network too. And make turns the host name into an address inside the container, when it writes the guest's files. So 127.0.0.1 is wrong twice: for image-garden it's the container, and for the guest it's the guest itself. Even 10.0.2.2, the trick that works on a host, only reaches the container. Give the pkgproxy machine's address on your network, or a name that resolves to it inside the container.

Details and sources

Details

  • make resolves the URL's host with getent ahostsv4 where make runs, here inside the container, and writes the address as pkgproxy in the guest's /etc/hosts; a host that does not resolve gives cannot use pkgproxy at "…": want http://host[:port] with a host that resolves to IPv4; guests get no pkgproxy (mk/010-qemu.mk:24-29).
  • Checked by running v0.8.0's image-garden make ubuntu-cloud-24.04.x86_64.user-data outside the snap: with PKGPROXY_URL=http://localhost:3142 the user-data adds 127.0.0.1 pkgproxy to /etc/hosts; with a name that does not resolve it has no pkgproxy lines and make prints the warning above.
  • image-garden's own downloads go to the URL from the container (image-garden:281-298).
  • 10.0.2.2 is the host's loopback as a guest of QEMU's user network sees it (tests/smoke-pkgproxy.sh:229-233). Derived: when QEMU runs in a container, that host is the container, so 10.0.2.2 reaches only what listens on the container's loopback, and image-garden in the container cannot use that address at all.
  • Derived: the address must route from the container's network; with Docker's default bridge network that is the pkgproxy machine's address on your network. The craft components' README says the same of Docker's host-gateway (ext-craft-pkgproxy).

03 Images one way, guests another

IMAGES AND GUESTS APART images guests PKGPROXY_URL=http://A:3142 pkgproxy A pkgproxy A + GARDEN_DL_PKGPROXY=http://B:3142 pkgproxy B pkgproxy A + GARDEN_DL_PKGPROXY= the origin pkgproxy A + GARDEN_DL_PKGPROXY=unix:/run/pkgproxy.sock -v HOST-SOCKET:/run/pkgproxy.sock the socket pkgproxy A guests follow PKGPROXY_URL only
With PKGPROXY_URL alone, images and guests use pkgproxy A. Adding GARDEN_DL_PKGPROXY=http://B:3142 sends images to pkgproxy B; setting it empty sends images to their origin; unix:/run/pkgproxy.sock with the socket mounted sends images through the socket. In every case the guests use pkgproxy A, from PKGPROXY_URL.

GARDEN_DL_PKGPROXY overrides PKGPROXY_URL for images only: another URL, empty for direct downloads, or unix:/path/pkgproxy.sock for a mounted socket. Guests follow PKGPROXY_URL.

Sometimes you want the two apart. GARDEN_DL_PKGPROXY names a pkgproxy for image downloads alone, and when it's set, it wins over PKGPROXY_URL. Set it to another URL, and images come from that proxy while the guests still use the first. Set it empty, and images download straight from their origin. It also takes the unix: form, the path of a pkgproxy socket. Mount the socket of a pkgproxy on the host into the container, and image downloads go through it with no network address at all. Guests can't use a socket here, so they still need the URL, or go without.

Details and sources

Details

  • When GARDEN_DL_PKGPROXY is set, even empty, image-garden does not take PKGPROXY_URL for downloads (image-garden:31-38); empty turns downloads through pkgproxy off (README.md:246-248).
  • unix:PATH is reached with curl --unix-socket PATH and the URL http://pkgproxy/… (image-garden:121-136, image-garden:297-298); the README names the form (README.md:273-278). A socket is never remembered as unreachable: connecting to it fails at once (image-garden:149).
  • Guests follow only PKGPROXY_URL outside the snap (mk/010-qemu.mk:20-44).
  • Derived, not run: a host's pkgproxy socket can be bind-mounted into the container with -v; the path /run/pkgproxy.sock and HOST-SOCKET on the slide are placeholders. The user in the container needs permission to connect to it.
  • A and B stand for the addresses of two pkgproxies.

04 What a container forgets

A SHORT MEMORY job 1's container image-garden pkgproxy, down one try, 10 s to connect the image's origin /run/image-garden/pkgproxy-unreachable.HASH skip pkgproxy for five minutes in /run: a container has no XDG_RUNTIME_DIR the container exits: the note is gone job 2's container: tries pkgproxy again /run not writable: nothing is remembered a guest one probe, then uncached
In job 1's container, image-garden tries the pkgproxy that is down once, with a 10 second connect timeout, then downloads from the image's origin and writes /run/image-garden/pkgproxy-unreachable.HASH for five minutes, in /run because no XDG_RUNTIME_DIR is set. When the container exits the note is gone, so job 2's container tries again. Where /run cannot be written nothing is remembered. A guest probes once in its first boot and boots uncached.

An unreachable pkgproxy is remembered for five minutes in /run/image-garden, inside the container, and forgotten when it exits. Each new container tries once, with a 10 second connect timeout.

When the proxy is down, the container shows its short memory. A download tries the proxy once, gives up after ten seconds without a connection, and goes straight to the origin. image-garden then remembers, for five minutes, not to ask again. In the container, that note lives in /run/image-garden, because no XDG_RUNTIME_DIR is set, and it's gone when the container exits. So every new job waits for it once more. Run as a user who can't write to /run, and nothing is remembered at all. The guest pays on its own: one short probe in its first boot, and then it boots uncached.

Details and sources

Details

  • An image is asked of pkgproxy once, with --connect-timeout 10; on failure "Cannot download through pkgproxy, downloading URL directly" (image-garden:287-307).
  • When curl could not resolve, connect or connect in time (exit 6, 7 or 28), image-garden writes $XDG_RUNTIME_DIR/image-garden/pkgproxy-unreachable.HASH and says "Not asking pkgproxy at … again for 5 minutes"; for 300 seconds each download says "Skipping pkgproxy at …, unreachable recently, downloading … directly" (image-garden:138-193).
  • Without XDG_RUNTIME_DIR the directory is /run, "as for root in a container"; where /run/image-garden cannot be made, nothing is remembered (image-garden:61-64, image-garden:138-143, README.md:254-258).
  • Derived: /run in a container is part of the container, so the note goes with it; a new container starts without it.
  • A guest asks pkgproxy once in bootcmd and skips its setup scripts in that boot when nothing answers (mk/010-qemu.mk:122-126).

05 Where to go from here

ROADS From a container Reaching pkgproxy every way a guest finds it Image downloads by their checksum Nobody answers an unreachable pkgproxy Changing the proxy prepared images keep theirs In CI the runner's environment The craft components the same rules in their jobs
Six roads from pkgproxy from a container: reaching pkgproxy, image downloads, an unreachable pkgproxy, changing the proxy, the container in CI, and the craft components

Next: how a guest reaches pkgproxy, image downloads, an unreachable pkgproxy, changing the proxy, the runner's environment, and the craft components.

So in the container, pkgproxy is a URL that works from the container and from the guests inside it, with GARDEN_DL_PKGPROXY when images should go another way. Next, the reaching deck shows every way a guest finds pkgproxy, and the addresses that point back. The downloads deck shows how images come through it by checksum. Then what happens when nobody answers, and why a prepared image keeps the proxy it was made with. In CI, set the URL once, in the runner's environment; the craft components' jobs follow the same rules.

Details and sources

Read more

Basis of the statements on this page

Read in image-garden, mk/010-qemu.mk, the README and proxy/README.md of image-garden v0.8.0 ("code"). The guest addresses were checked by running image-garden make ubuntu-cloud-24.04.x86_64.user-data outside the snap. The container commands below were not run for this map; what follows from container networking is "derived".

The variables

VariableRead byEffect in the container
PKGPROXY_URLimage-garden and makeimage downloads, unless GARDEN_DL_PKGPROXY is set; guests' apt, snap and Go setup
GARDEN_DL_PKGPROXYimage-gardenimage downloads only: a URL, unix:PATH, or empty for none

Examples

# one pkgproxy for everything docker run --rm -it --device /dev/kvm \ -v $HOME/.cache/image-garden:/var/cache/image-garden \ -e PKGPROXY_URL=http://pkgproxy.lan:3142 \ registry.gitlab.com/zygoon/image-garden:TAG \ image-garden make ubuntu-cloud-24.04.x86_64.run

# guests through the URL, images through a mounted socket -v /path/to/pkgproxy.sock:/run/pkgproxy.sock \ -e PKGPROXY_URL=http://pkgproxy.lan:3142 \ -e GARDEN_DL_PKGPROXY=unix:/run/pkgproxy.sock

# guests through the URL, images straight from their origin -e PKGPROXY_URL=http://pkgproxy.lan:3142 \ -e GARDEN_DL_PKGPROXY=

Addresses

Address in PKGPROXY_URLFrom image-garden in the containerFrom the guest
127.0.0.1 or localhostthe containerthe guest itself
10.0.2.2not reachablethe container's loopback
the pkgproxy machine's LAN address or namethe pkgproxythe pkgproxy

Roads from here

BUILT ON IT · CI integrations

PKGPROXY · caching for guests

CONTAINER · OCI image