LXD build containers
lxd for users for administrators for CI operators
pkgproxyctl setup lxd and setup snapcraft, rockcraft or charmcraft put apt, snapd and Go inside LXD build containers behind pkgproxy, with a self-contained profile and printed scripts that run as you.
On the map: lxd
01 What this topic covers
setup lxd and the craft setups put apt, snapd and Go inside
LXD build containers behind pkgproxy.
LXD build containers are where your builds run, set up so that apt, snapd and Go inside them download through pkgproxy. If you have not seen LXD images yet, start there. First you see the two ways LXD uses pkgproxy, the profile that holds everything, and which profile it changes. Administrators and CI operators also see the address the containers use, and administrators why the script runs as you. Last, you see setup and undo run twice.
02 Two directions
setup lxd-remote serves LXD its images;
setup lxd points apt, snapd and go inside the containers at
pkgproxy. A craft build needs setup lxd-remote and
setup snapcraft (or rockcraft,
charmcraft).
LXD can use pkgproxy in two ways. setup lxd-remote makes pkgproxy the image server LXD launches containers from. Setup lxd works the other way round: it changes an LXD profile so that what runs inside each container, apt, snapd and the Go command, downloads through pkgproxy too. A build in a container then gets both its image and its packages from the cache. A snapcraft, rockcraft or charmcraft build needs both commands.
03 Everything in the profile
environment.GOPROXY and
environment.SNAPPY_FORCE_API_URL, plus raw.lxc
pre-start and post-stop hooks that write and remove
99pkgproxy-lxd and pkgproxy-lxd.conf. Each hook
always exits 0 and never follows a symbolic link.
Everything the containers need is in the LXD profile itself, so nothing is installed in them or on the LXD host. Two environment keys point the Go command, and the snap command that fetches stage-snaps, at pkgproxy. Two LXC hooks do the rest. Before a container starts, one writes apt and snapd configuration into its root file system, and switches Ubuntu's archives from HTTPS to HTTP so that their packages can be cached. After the container stops, the other removes those two files.
04 A profile of its own, or a project's default
Without -project: a standalone pkgproxy profile
(lxc launch … --profile pkgproxy --profile default). With
-project P: a PATCH of P's default profile. setup
snapcraft, rockcraft and charmcraft fix P
to the tool's own project.
Setup lxd with no project makes a separate profile named pkgproxy, which you add to a container next to the default one. With the project option it changes that project's default profile instead, so every container in the project is covered. Setup snapcraft, rockcraft and charmcraft are that, for the project each tool builds in. If the tool has not run yet, the script creates its project the way the tool would. It also deletes the tool's base instances, so the next build starts from one with pkgproxy in place.
05 The address the containers use for administrators for CI operators
The host of -addr (or of the address the setup page was
reached at): an IP is used as is, a name is resolved with
getent, and none, a wildcard, localhost or
loopback means the managed LXD bridge's ipv4.address (else
ipv6.address).
The containers must reach pkgproxy at an address that works from inside them, and the script works it out on the machine that runs it. An IP address in the -addr flag is used as it is. A host name is resolved there, preferring IPv4, because the containers have neither that machine's hosts file nor its resolver. With no host, or a loopback one, pkgproxy runs on this machine, and the containers reach it on their profile's bridge, one that LXD manages, at the bridge's own address.
06 The script runs as you, not in the daemon for administrators
The LXD calls stay in the printed script, run by the user in the
lxd group with lxc … local:. The admin socket,
-admin-socket-path, serves only DELETE
/pkgproxy/v1/cache and /pkgproxy/v1/offline.
pkgproxy and LXD are separate confined snaps, and pkgproxy has no access to LXD's socket. So the daemon never changes LXD: pkgproxyctl prints a script, and your own shell runs it with lxc, on the local remote. That needs membership in the lxd group, not root. The daemon does have a socket for changes, the admin socket, but it only takes cache invalidate and the offline mode, and it is kept apart from the read only stats socket.
07 Setup twice, undo twice
-undo deletes the pkgproxy profile, or unsets
environment.GOPROXY, environment.SNAPPY_FORCE_API_URL
and raw.lxc on a project's default profile. Setup and undo can each be
run again safely, and a second snapcraft pack is served from
the cache.
Running setup a second time changes nothing more. Undo removes what setup added: the separate profile is deleted, and a project's default profile loses the two environment keys and raw.lxc, with any value of your own under those keys. A running container keeps the hooks it started with, so its post-stop hook still removes the files when it stops. Undo is safe to repeat and to run before setup. With both setups in place, a second snapcraft pack gets its image, snaps, debs and Go modules from the cache.
08 Where to go next for CI operators in place of the general version
Next: pkgproxy for CI, for many build machines behind one cache; Offline mode, for builds that see only what is cached.
Builds in your LXD containers now get their packages from pkgproxy, and with both setups a second craft build is served from the cache. To put many build machines behind one cache, watch pkgproxy for CI. To run builds that see only what is cached, watch Offline mode.
09 Where to go next general version
Next: LXD images, for the images the containers start from; pool and dists, for what apt gets from the cache; Offline mode, for building with no network.
Builds in your LXD containers now get their packages from pkgproxy, and with both setups a second craft build is served from the cache. To see the images the containers start from, watch LXD images. To see what apt in the container gets from the cache, watch pool and dists. To build with no network at all, watch Offline mode.
Read more
Roads from here
CORE · the path every side shares
- Offline mode — building with no network
APT
- pool forever and dists rechecked — what apt in the container gets from the cache
GO MODULES
- Go module proxy — where GOPROXY in the container points
LXD
- LXD image servers — the images the containers start from
SETUP AND THE COMMAND LINE
- Setting up clients — the other setup commands and -undo
OPERATIONS
- pkgproxy for CI — many build machines behind one cache