LXD build containers

lxd for users for administrators for CI operators

pkgproxyctl setup lxd and setup snapcraft, rockcraft or charmcraft put apt, snapd and Go inside LXD build containers behind pkgproxy, with a self-contained profile and printed scripts that run as you.

On the map: lxd

▶ Watch this deck — 9 scenes, ~5 min

01 What this topic covers

ABOUT THIS TOPIC LXD build containers apt, snapd and Go inside LXD containers, behind pkgproxy 1 Two directions 2 Everything in the profile 3 A profile of its own, or a project's default 4 The address the containers use 5 The script runs as you, not in the daemon 6 Setup twice, undo twice
The topic title, LXD build containers, and its six scenes in order: two directions, the profile, a profile or a project, the address, the script runs as you, setup and undo

setup lxd and the craft setups put apt, snapd and Go inside LXD build containers behind pkgproxy.

LXD build containers are where your builds run, set up so that apt, snapd and Go inside them download through pkgproxy. If you have not seen LXD images yet, start there. First you see the two ways LXD uses pkgproxy, the profile that holds everything, and which profile it changes. Administrators and CI operators also see the address the containers use, and administrators why the script runs as you. Last, you see setup and undo run twice.

02 Two directions

TWO DIRECTIONS LXD pkgproxy image :3143 setup lxd-remote container apt snapd go debs, snaps, modules setup lxd :3142 $ pkgproxyctl setup lxd-remote | sh $ pkgproxyctl setup snapcraft | sh a craft build needs both
pkgproxy serves an LXD host its container image over port 3143, set up by setup lxd-remote; apt, snapd and go inside a container download through pkgproxy on port 3142, set up by setup lxd; a craft build runs both commands

setup lxd-remote serves LXD its images; setup lxd points apt, snapd and go inside the containers at pkgproxy. A craft build needs setup lxd-remote and setup snapcraft (or rockcraft, charmcraft).

LXD can use pkgproxy in two ways. setup lxd-remote makes pkgproxy the image server LXD launches containers from. Setup lxd works the other way round: it changes an LXD profile so that what runs inside each container, apt, snapd and the Go command, downloads through pkgproxy too. A build in a container then gets both its image and its packages from the cache. A snapcraft, rockcraft or charmcraft build needs both commands.

03 Everything in the profile

THE PROFILE config: environment.GOPROXY: http://10.0.0.1:3142/go,direct environment.SNAPPY_FORCE_API_URL: http://10.0.0.1:3142 raw.lxc: |- lxc.hook.pre-start = /bin/sh -c "echo … | base64 -d | sh -s 10.0.0.1:3142" lxc.hook.post-stop = /bin/sh -c "echo … | base64 -d | sh -s" pre-start /etc/apt/apt.conf.d/99pkgproxy-lxd snapd.service.d/pkgproxy-lxd.conf https://archive.ubuntu.com → http:// so that the container's debs can be cached post-stop removes the two files the http:// archives stay: they work without pkgproxy too
The LXD profile sets GOPROXY and SNAPPY_FORCE_API_URL and two raw.lxc hooks; the pre-start hook writes apt and snapd configuration into the container and rewrites https Ubuntu archives to http; the post-stop hook removes the two files

environment.GOPROXY and environment.SNAPPY_FORCE_API_URL, plus raw.lxc pre-start and post-stop hooks that write and remove 99pkgproxy-lxd and pkgproxy-lxd.conf. Each hook always exits 0 and never follows a symbolic link.

Everything the containers need is in the LXD profile itself, so nothing is installed in them or on the LXD host. Two environment keys point the Go command, and the snap command that fetches stage-snaps, at pkgproxy. Two LXC hooks do the rest. Before a container starts, one writes apt and snapd configuration into its root file system, and switches Ubuntu's archives from HTTPS to HTTP so that their packages can be cached. After the container stops, the other removes those two files.

04 A profile of its own, or a project's default

PROFILE OR PROJECT $ pkgproxyctl setup lxd | sh profile "pkgproxy" container a container b --profile pkgproxy --profile default $ pkgproxyctl setup lxd -project P | sh project P: default profile container a container b every container in the project the craft tools, each in its own project: setup snapcraft setup rockcraft setup charmcraft creates the project if the tool has not run yet *-base-instance-* deleted: the next build starts from one with pkgproxy in place
setup lxd without a project makes a pkgproxy profile added to chosen containers; with -project it patches that project's default profile, which setup snapcraft, rockcraft and charmcraft do for the snapcraft, rockcraft and charmcraft projects; the craft tools' base instances are deleted

Without -project: a standalone pkgproxy profile (lxc launch … --profile pkgproxy --profile default). With -project P: a PATCH of P's default profile. setup snapcraft, rockcraft and charmcraft fix P to the tool's own project.

Setup lxd with no project makes a separate profile named pkgproxy, which you add to a container next to the default one. With the project option it changes that project's default profile instead, so every container in the project is covered. Setup snapcraft, rockcraft and charmcraft are that, for the project each tool builds in. If the tool has not run yet, the script creates its project the way the tool would. It also deletes the tool's base instances, so the next build starts from one with pkgproxy in place.

05 The address the containers use for administrators for CI operators

WHICH ADDRESS -addr given to pkgproxyctl address in the profile -addr 192.0.2.10:3142 used as it is 192.0.2.10:3142 -addr cache.lan:3142 getent, IPv4 first its address containers have neither this machine's /etc/hosts nor its resolver :3142 or localhost pkgproxy is here lxdbr0's ipv4.address the bridge on the profile's nic managed: true, type: bridge never Docker's docker0
Three cases for the address in the profile: an IP address is used as is; a host name is resolved on the machine running the script; no host or a loopback address means the LXD bridge's own address

The host of -addr (or of the address the setup page was reached at): an IP is used as is, a name is resolved with getent, and none, a wildcard, localhost or loopback means the managed LXD bridge's ipv4.address (else ipv6.address).

The containers must reach pkgproxy at an address that works from inside them, and the script works it out on the machine that runs it. An IP address in the -addr flag is used as it is. A host name is resolved there, preferring IPv4, because the containers have neither that machine's hosts file nor its resolver. With no host, or a loopback one, pkgproxy runs on this machine, and the containers reach it on their profile's bridge, one that LXD manages, at the bridge's own address.

06 The script runs as you, not in the daemon for administrators

WHO TALKS TO LXD pkgproxy confined snap LXD unix.socket no access $ pkgproxyctl setup lxd | sh your shell lxc … local: needs the lxd group, not root admin.sock cache invalidate · offline mode stats.sock read only: stats, info two sockets, two kinds of trust
The pkgproxy snap has no path to LXD's socket; pkgproxyctl prints a script that the user's shell runs with lxc against LXD; the daemon's admin socket takes only cache invalidation and the offline mode, separate from the read-only stats socket

The LXD calls stay in the printed script, run by the user in the lxd group with lxc … local:. The admin socket, -admin-socket-path, serves only DELETE /pkgproxy/v1/cache and /pkgproxy/v1/offline.

pkgproxy and LXD are separate confined snaps, and pkgproxy has no access to LXD's socket. So the daemon never changes LXD: pkgproxyctl prints a script, and your own shell runs it with lxc, on the local remote. That needs membership in the lxd group, not root. The daemon does have a socket for changes, the admin socket, but it only takes cache invalidate and the offline mode, and it is kept apart from the read only stats socket.

07 Setup twice, undo twice

UNDO $ pkgproxyctl setup snapcraft | sh # twice = once $ pkgproxyctl setup snapcraft -undo | sh profile "pkgproxy" deleted environment.GOPROXY environment.SNAPPY_FORCE_API_URL raw.lxc unset without -project a project's default profile: your own values go too running container post-stop hook cleans up undo twice, or before setup: no change snapcraft pack second time: image, snaps, debs and Go modules from the cache hit
setup run twice equals setup once; -undo deletes the pkgproxy profile or unsets three keys of a project's default profile; a running container's post-stop hook removes the files when it stops; a second snapcraft pack is served from the cache

-undo deletes the pkgproxy profile, or unsets environment.GOPROXY, environment.SNAPPY_FORCE_API_URL and raw.lxc on a project's default profile. Setup and undo can each be run again safely, and a second snapcraft pack is served from the cache.

Running setup a second time changes nothing more. Undo removes what setup added: the separate profile is deleted, and a project's default profile loses the two environment keys and raw.lxc, with any value of your own under those keys. A running container keeps the hooks it started with, so its post-stop hook still removes the files when it stops. Undo is safe to repeat and to run before setup. With both setups in place, a second snapcraft pack gets its image, snaps, debs and Go modules from the cache.

08 Where to go next for CI operators in place of the general version

WHERE NEXT builds in your LXD containers get their packages from pkgproxy; with both setups, a second craft build is served from the cache you are here LXD build containers pkgproxy for CI many build machines behind one cache Offline mode builds that see only what is cached
You are here, LXD build containers; next: pkgproxy for CI, Offline mode

Next: pkgproxy for CI, for many build machines behind one cache; Offline mode, for builds that see only what is cached.

Builds in your LXD containers now get their packages from pkgproxy, and with both setups a second craft build is served from the cache. To put many build machines behind one cache, watch pkgproxy for CI. To run builds that see only what is cached, watch Offline mode.

09 Where to go next general version

WHERE NEXT builds in your LXD containers get their packages from pkgproxy; with both setups, a second craft build is served from the cache you are here LXD build containers LXD images the images the containers start from pool and dists what apt in the container gets from the cache Offline mode building with no network at all
You are here, LXD build containers; next: LXD images, pool and dists, Offline mode

Next: LXD images, for the images the containers start from; pool and dists, for what apt gets from the cache; Offline mode, for building with no network.

Builds in your LXD containers now get their packages from pkgproxy, and with both setups a second craft build is served from the cache. To see the images the containers start from, watch LXD images. To see what apt in the container gets from the cache, watch pool and dists. To build with no network at all, watch Offline mode.

Read more

Roads from here

CORE · the path every side shares

APT

GO MODULES

LXD

SETUP AND THE COMMAND LINE

OPERATIONS