pkgproxySnap StoreSource
Contents · Integration tests

Integration tests

The unit tests (./check.sh) cover pkgproxy’s own logic. The spread suite in tests/ covers what they cannot: the snap, installed the way users install it, used by the real apt, snapd, Docker, Go and LXD of an Ubuntu 26.04 VM.

Each test asserts behaviour, never fixed archive content: a second download is a cache hit (from the stats API), clients reach pkgproxy (from its access log), and the pkgproxyctl setup commands do what they say, as often as they are run. After every test, tests/lib/invariants.sh checks that no proxy setting is left on the machine, so that no test depends on what another left behind.

Prepare pins auto-offline=off and offline-mode=never, so that a VM whose network blinks cannot take pkgproxy offline under a test; the offline tests below change them, and put them back.

Layout

There is one suite for each kind of client, tests/<client>/: apt, snap, docker, go, lxd-remote and lxd, plus tests/install/ for the snap itself: its daemon and sockets (daemon), and one variant of snap-option for each snap set pkgproxy option, which checks that a good value reaches the daemon and a bad one is refused. Every client suite has the same tasks about its setup command:

  • setup: the setup command, piped to sh, configures the client.
  • setup-setup: running it twice leaves the machine as running it once.
  • undo: setup, then setup -undo, leaves the machine as it was.
  • undo-undo: a second -undo succeeds and changes nothing more.
  • undo-first: -undo on a machine never set up succeeds and changes nothing.
  • smoke: the client downloads through pkgproxy, and a second download is a cache hit.

tests/snapcraft/pack is a round trip of its own: as a user, it runs setup lxd-remote and setup snapcraft as README.md says, then snapcraft pack of the project in tests/snapcraft/fixture, twice. The fixture’s build downloads one of each kind (the build image, a build snap and a stage snap, debs and Go modules), and the test checks that each came through pkgproxy and that the second pack, after the build instances and images are removed, gets each from the cache.

tests/charmcraft/pack is the same round trip for charmcraft, with setup lxd-remote and setup charmcraft, and the toy machine charm in tests/charmcraft/fixture. Its build fetches the build image, a build snap and debs, which must come through pkgproxy and, the second time, from its cache, and ops from PyPI, which cannot: PyPI is no upstream of pkgproxy, and uv fetches it over HTTPS, which pkgproxy only tunnels. The test checks that the containers reach PyPI directly, over the HTTPS its firewall lets through, and that pkgproxy sees none of it.

tests/rockcraft/pack does the same for rockcraft, with setup lxd-remote and setup rockcraft, and rockcraft pack of the rock in tests/rockcraft/fixture, twice. The rock is bare, with an Ubuntu 24.04 build base, so its build downloads the build image, a stage snap (pebble, which rockcraft adds to every rock) and debs (a stage package), all of which pkgproxy caches. A rock on any other base starts from an OCI image that rockcraft pulls from public.ecr.aws over HTTPS, which pkgproxy does not cache. Each task starts with no rockcraft LXD project, so that the setup makes it.

The first five run the wl_* functions of tests/lib/workload/<client>.sh: what setup and undo run, what they may change (wl_state, which the tasks compare before and after), and what a configured and an unconfigured machine look like. A suite’s restore-each in spread.yaml undoes its setup the way a user would. tests/lib/pkgproxy.sh holds the helpers every task uses.

Offline tests

Some suites have an offline task, and apt and snap a store-overloaded one too; tests/install/offline-switch covers pkgproxyctl offline, and tests/install/auto-offline going offline with the machine. Each task warms the cache itself, online, so that it depends on no other task, and asks for something no task fetches (or, for apt, that pkgproxyctl cache invalidate threw away) to see a refusal. It switches with snap set pkgproxy offline-mode=..., as a user would.

While offline, a leak guard watches the daemon: an nftables table, inet pkgproxy_spread, that matches what the daemon’s cgroup sends out, lets answers to clients through, counts DNS lookups and counts and rejects everything else. pp_assert_no_upstream then checks that every request since the switch was a hit, an answer synthesised from the cache or a refusal, and that the guard counted nothing. In store-overloaded mode the guard plays the upstream in trouble instead. A task’s restore puts the mode back to never and removes the guard, and invariants.sh checks both, along with auto-offline=off and the default routes, putting each right before it fails.

Writing a test

Spread runs every script with set -eu; tests/lib/pkgproxy.sh adds pipefail, and a script that does not source it sets that itself. Three habits follow:

  • Check a negative with pp_not: spread ignores a failing ! cmd.
  • Capture a probe before testing what it printed, out="$(probe)", so that set -e stops the test when the probe fails, rather than taking its empty output for a clean result.
  • Write cmd | grep PATTERN >/dev/null, not cmd | grep -q PATTERN: with pipefail, grep -q can fail the pipeline by stopping early.

A check that something came from the cache looks at content, not metadata: pkgproxy also counts the lists and catalogs it serves within their TTL as hits. pp_content_re gives the stats keys of each kind’s content.

Who runs what

Spread runs every script as root. The setup commands run as the documentation says: setup apt, setup snap and setup docker as root, and setup go, setup lxd-remote, setup lxd, setup snapcraft, setup charmcraft, setup rockcraft, snapcraft, charmcraft and rockcraft as the unprivileged user ubuntu. setup lxd-remote needs that: it changes the user’s own lxc remotes, and refuses to run under sudo. So does setup go, which changes the user’s own go env file. pp_user runs a command as that user from the root script, in a login shell made by runuser: with the user’s HOME, PATH and groups (lxd included), and none of root’s environment. The rest of a task, such as reading the stats socket or the journal, stays root.

What it needs

  • image-garden, which makes and starts the VM (the garden backend of spread.yaml), and spread. The image-garden snap comes with both.
  • KVM (/dev/kvm), and about 6 GiB of memory for each VM. Spread starts eight at once (workers in spread.yaml), each with its own pkgproxy, and runs the suites in parallel.
  • Network access. The tests download packages, snaps, Docker Hub images and LXD images. Docker Hub limits how many images an address may pull without logging in.

The first run downloads the Ubuntu 26.04 cloud image and makes a VM image from it, with the packages .image-garden.mk lists. Later runs start from that image, until .image-garden.mk changes. image-garden keeps the images, logs and run state in .image-garden/, which git and spread ignore. Delete the image there, or in image-garden’s download cache, to start from a newer cloud image.

Building the snap

The suite tests the one pkgproxy_*.snap in the root of the tree, and refuses to run with none or more than one. It installs the snap with --dangerous, which gets none of the aliases or auto-connections the store grants, so the suite’s prepare makes the pkgproxyctl alias and connects the plugs itself. Build the snap on the host:

rm -f pkgproxy_*.snap
snapcraft pack

snapcraft pack does not work in a git worktree. Build in the main checkout, or copy the snap built there into the worktree.

Running

spread -list
spread garden:ubuntu-cloud-26.04
spread garden:ubuntu-cloud-26.04:tests/apt/
spread garden:ubuntu-cloud-26.04:tests/apt/smoke

spread -debug opens a shell in the VM when a test fails, and spread -reuse keeps the VMs for the next run. With eight VMs the whole suite takes about 15 minutes, most of it for the snapcraft and charmcraft packs.