How pkgproxy is tested

contributors for contributors

Unit tests of pure logic, command tests through the real command tree, synctest bubbles instead of sleeps, exploratory runs of real clients and the spread suite in VMs, all behind ./check.sh and a pre-commit hook.

On the map: inside pkgproxy

▶ Watch this deck — 10 scenes, ~6 min

01 What this topic covers

ABOUT THIS TOPIC How pkgproxy is tested from unit tests of pure logic to real clients in virtual machines 1 Pure logic, unit tested 2 Commands as a user runs them 3 No sleeps, a bubble 4 Real clients, by hand 5 The spread suite asserts behaviour 6 Upstreams that fail on purpose 7 The rock and the charms 8 Before a commit, before a push
About this topic, How pkgproxy is tested: pure logic, command tests, synctest, real clients, the spread suite, upstreams that fail on purpose, the rock and the charms, before a commit

Unit tests, command tests, synctest bubbles, exploratory runs, the spread suite, outages on purpose, the rock and the charms, and the checks before a commit.

pkgproxy is tested at several levels, from unit tests of pure logic to real clients in virtual machines. If you have not seen The source tree yet, start there. You will see unit tests of pure logic, and command tests that run the real command tree. Then come synctest bubbles that replace sleeps, and real clients tried by hand. Then come the spread suite in virtual machines, upstreams that fail on purpose, and the tests of the rock and the charms. Last, what runs before a commit and before a push.

02 Pure logic, unit tested

UNIT TESTS index parsing cache keys hash checks store on disk no network: the same result every time $ ./check.sh dagger check go test -race -short ./... pinned + latest Go fmt vet golint build staticcheck goconst errcheck wsl ineffassign reuse GitLab CI: the same Dagger module
Four kinds of pure logic are unit tested without a network; ./check.sh runs dagger check, the same module GitLab CI runs, with the race-detected tests and the linters

Index parsing, cache keys, hash checks and the Store are plain unit tests. ./check.sh is dagger check: fmt, vet, golint, build, staticcheck, goconst, errcheck, wsl, ineffassign, go test -race -short on two Go versions, and reuse lint.

Most of pkgproxy is pure logic, and it is unit tested like any Go code: index parsing, cache key normalization, hash verification, and the store interface against the local file system. Those tests touch no network, so they give the same result every time. ./check.sh runs them with every other check as a dagger check: go test with the race detector, on the pinned Go and the latest one, next to the linters. GitLab CI runs the same Dagger module.

03 Commands as a user runs them

COMMAND TESTS func TestCtlSetupLxdRemote(t *testing.T) { noDaemon(t) // SNAP_COMMON: an empty directory inv := invoke(t, "pkgproxyctl", "setup", "lxd-remote", "-addr", testAddr, "-lxd-tls-addr", ":8443", …) check(t, inv.ExpectExitCode(0), inv.ExpectStdout(`<IGNORE> { …the whole piped script, line by line… } `), inv.ExpectStderr("")) } unixServer(t, handler) a fake daemon on a UNIX socket, for stats, info and admin calls
A Go test calls noDaemon, then invoke with pkgproxyctl setup lxd-remote and its flags, and checks the exit code and the whole printed script; unixServer can stand in for a daemon on a socket

invoke(t, "pkgproxyctl", "setup", …) wraps go-cmdr's cmdtest; assert the whole stdout, with <IGNORE> where it varies. noDaemon(t) points SNAP_COMMON at an empty directory; unixServer fakes a daemon.

pkgproxy and pkgproxyctl are tested the way a user runs them, through the real command tree. The invoke helper wraps the cmdtest package: it runs the command with its arguments, and its output buffers make every setup command print its piped script. The test then compares the whole output. Calling noDaemon first keeps a pkgproxy running on the test machine out of the output, and unixServer serves a fake daemon on a socket when a test needs one.

04 No sleeps, a bubble

SYNCTEST time.Sleep(100 * time.Millisecond) synctest.Test(t, func(t *testing.T) { … }) request 1 request 2 request 3 one fetch synctest.Wait() returns once all of them are blocked fake clock +30s at once, exactly httptest .NewTestServer in memory a real socket stops the clock a goroutine blocked on it is not "blocked" to synctest
Inside a synctest bubble, three requests join one fetch and synctest.Wait returns once they are all blocked; the fake clock jumps a timeout exactly; the upstream is an in-memory httptest server, while a real socket would stop the clock

synctest.Test runs the test in a bubble; synctest.Wait() returns when everything else is blocked. Timeouts and TTLs pass on a fake clock. Use httptest.NewTestServer and its Client(), not a real socket.

A test never waits for a goroutine with a sleep or a polling loop. It runs in a synctest bubble instead. synctest.Wait returns once every other goroutine in the bubble is blocked, such as requests that joined a shared fetch. Time in the bubble is a fake clock, so a thirty second idle timeout passes at once, and exactly. Anything the test talks to over HTTP is on httptest's in memory network, because a goroutine blocked on a real socket keeps the clock from moving.

05 Real clients, by hand

EXPLORATORY podman: debian or ubuntu $ apt update $ apt install hello $ apt download hello pkgproxy on the host deb.debian.org changes over time, not ours to pin a bug a narrow unit test: "this Release stanza parses wrong" not a frozen end-to-end scenario
A podman container runs apt update, install and download through pkgproxy on the host to a real mirror; a bug found that way turns into a narrow unit test, not a frozen end-to-end test

Exploratory apt runs in a podman container against the pkgproxy on the host; no CI job snapshots a mirror. A bug found this way becomes a narrow unit test of the logic that failed.

Interoperability with apt is checked by hand, not by a test pinned to an archive. Run a Debian or Ubuntu container with podman, point its apt at the pkgproxy on the host, and run apt update, apt install and apt download through it. Archives change over time and their mirrors are not ours to pin, so this stays exploratory. When it finds a bug, the fix comes with a narrow unit test of the logic underneath, such as a Release stanza that parsed wrongly.

06 The spread suite asserts behaviour

SPREAD ubuntu-cloud-26.04-at-lxd5 ubuntu-cloud-26.04-at-lxd6 image-garden VMs, LXD 5.21 and LXD 6 CI runs this one tests/apt/ · snap/ · docker/ · go/ · lxd/ · lxd-remote/ setup setup-setup twice is once undo undo-undo undo-first smoke download through pkgproxy, then again: hit behaviour, never archive content tests/lib/invariants.sh after every task: no proxy setting left on the machine
Each client suite of the spread suite has the same six tasks, run in image-garden VMs with LXD 5.21 and LXD 6, and invariants.sh checks after every task that no proxy setting is left

tests/<client>/: setup, setup-setup, undo, undo-undo, undo-first, smoke. tests/lib/invariants.sh runs after every task. CI runs the LXD 6 system; LXD 5.21 is left to local runs.

The spread suite tests the snap with the real apt, snapd, Docker, Go and LXD, in Ubuntu virtual machines that image-garden makes, with LXD 5.21 and LXD 6. Every client suite has the same tasks: setup, setup twice, undo, undo twice, undo first, and a smoke test where the second download is a cache hit. The tests assert behaviour, never archive content. After each task, an invariants script checks that no proxy setting was left behind.

07 Upstreams that fail on purpose

OUTAGES, ON PURPOSE pkgproxy upstreams pp_guard_on rejectrefused at once: a server that is down dropno answer: a server that hangs tests/apt/upstream-silent · tests/snap/store-silent · tests/snap/store-down apt and snapd keep working from the cache pp_assert_served_locally # hit · stale · synthesised $ spread garden:ubuntu-cloud-26.04-at-bare:tests/offline-build/pack # online round fills the cache; snapd purged; # offline round: apt, snapd, LXD, snapcraft pack, from pkgproxy alone
The leak guard, an nft rule on pkgproxy's cgroup, rejects connections as a server that is down, or drops packets as a silent server; the upstream-silent and store-silent tasks keep apt and snapd working from the cache, checked by pp_assert_served_locally; tests/offline-build packs a snap on a fresh machine with only pkgproxy

pp_guard_on [all] [reject|drop]; pp_assert_served_locally accepts hit, stale and synthesised. tests/offline-build/pack is manual, about an hour, on ubuntu-cloud-26.04-at-bare, with pkgproxy as pkgproxy-offline.service.

Some tasks make the upstreams fail on purpose, the way real outages look to a client. A leak guard in the firewall stops pkgproxy's own traffic: it can reject each connection at once, as a server that is down, or drop the packets, so that a connection hangs, as a silent server does. The silent archive and silent store tasks check that apt and snapd keep working from the cache, and a helper asserts that every request was a hit, a stale copy or a placeholder. The offline build suite goes furthest: a fresh machine packs a snap with only pkgproxy to talk to.

08 The rock and the charms

THE ROCK AND THE CHARMS tests/rock/ rock in docker snap apt a named volume, then a new container: still a hit options tests/charm/ manual concierge: Juju · MicroK8s · LXD integration machine jubilant: deploy, apt twice, options spread-charm Juju 3.6/stable and 4/stable on main, on tags, on a merge request labelled spread-charm ./check.sh charm-lint mypy --strict charm-test pytest, both charms
Spread runs the rock in docker beside the snap; the manual charm suite sets up Juju, MicroK8s and LXD with concierge and runs jubilant tests of both charms; CI runs it on Juju 3.6 and 4 when asked; ./check.sh type-checks and unit-tests the charms' Python

tests/rock/: apt, options. tests/charm/ (manual): integration on MicroK8s, machine on LXD. The spread-charm job runs it with PKGPROXY_JUJU_CHANNEL 3.6/stable and 4/stable. ./check.sh runs charm-lint and charm-test with uv.

The rock and the charms have tests of their own. Spread runs the rock in docker beside the snap: apt downloads through it, and a new container finds the cache again on the same volume. The charm suite is manual, because its setup takes minutes: concierge installs Juju, a Kubernetes cluster, and a controller on the virtual machine's LXD. Then jubilant tests deploy each charm, download with apt twice, and change its options. CI runs that suite on Juju 3.6 and on Juju 4, on main, on tags, and on a merge request labelled spread-charm. ./check.sh adds mypy --strict and pytest for the charms' Python.

09 Before a commit, before a push

BEFORE A COMMIT $ git config core.hooksPath githooks once per clone git commit -s gofmt vet staticcheck goconst errcheck wsl ineffassign go mod tidy -diff, every module seconds, no container ./check.sh the tests, golint, reuse lint and the rest before a push GitLab CI dco-check fails a commit with no Signed-off-by
git commit runs the pre-commit hook's fast static checks and checks every module is tidy; ./check.sh runs the tests and the rest before a push; CI runs the same checks and dco-check, which needs a Signed-off-by line

git config core.hooksPath githooks enables githooks/pre-commit, which also runs go mod tidy -diff in every module. Tests, golint and reuse lint are left to ./check.sh and CI. Commit with git commit -s for the DCO check.

Enable the pre-commit hook in every checkout you commit from. It runs the static checks that take seconds, gofmt, vet, staticcheck, goconst, errcheck, wsl and ineffassign, through go tool, and checks that every module is tidy, with no container, so a lint failure stops the commit rather than a CI run. It does not run the tests: run ./check.sh, or at least go test of what you changed, before pushing. Every commit needs a Signed-off-by line, or CI fails it.

10 Where to go next general version

WHERE NEXT you know which kind of test a change needs, and what runs before you commit and push it you are here How it is tested internal/fill the code most synctest tests drive The source tree the conventions beyond the tests pkgproxy for CI the spread suite as a small CI farm
Where next from How pkgproxy is tested: internal/fill, The source tree, pkgproxy for CI

Next: internal/fill, for the code synctest tests drive; The source tree, for the conventions; pkgproxy for CI, for the spread suite as a farm.

You now know which kind of test a change needs, and what runs before you commit and push it. To see the fill code that many synctest tests drive, watch internal/fill. For the conventions a change follows beyond its tests, watch The source tree. To see the spread suite's setup as a small CI farm, watch pkgproxy for CI.

Read more

The kinds of test

KindWhereHow to run
Unit tests of pure logic*_test.go beside the code under internal/; fixtures in testdata/apt/go test ./internal/..., or one package
Command testscmd/pkgproxy/*_test.go, one beside each cmd_*.gogo test ./cmd/pkgproxy
Synctest tests (shared fetches, timeouts, TTLs)in the packages they test: fill, flight, idletimeout, proxy, snapproxy, lxdproxy, reqlog, cmd/pkgproxy and othersas their package
Documentation checksdocs/docs_test.go: TestTablesOfContents, TestLinks, TestNoRawTags, TestHeadergo test ./docs
Podman runs of real apt and gointernal/integration/go test ./internal/integration; skipped under -short, without podman, or when a container cannot reach the host
snapd's own store client against the snap proxy engineinternal/integration/snapd/, a nested modulecd internal/integration/snapd && go test ./...
Exploratory apt runsby hand: a Debian or Ubuntu container under podman with its apt pointed at the host's pkgproxynot automated; a bug found becomes a narrow unit test
Spread suitespread.yaml, tests/<suite>/<task>/task.yaml, tests/lib/spread garden: (below)
Charm unit testscharms/<name>/tests/unit/, with ops.testing, for pkgproxy-k8s and pkgproxyuv run pytest -q in charms/<name>/, or dagger check charm-test
Charm integration testscharms/<name>/tests/integration/, with jubilantthe manual spread suite tests/charm/ (below), or uv run pytest tests/integration against a Juju model of your own
Charms kept in step with the daemoncmd/pkgproxy/charm_test.gogo test ./cmd/pkgproxy
A benchmarkinternal/upstreams/history_test.go: BenchmarkHistory30Days, a series' 30 days of hours, the same cost whatever the files hold (§28.7)go test -run '^$' -bench History30Days ./internal/upstreams; neither ./check.sh nor CI runs it
Everything CI runs but spread.dagger/main.go./check.sh

Command test helpers

HelperFileWhat it does
TestMainmain_test.goStarted as pkgproxy or pkgproxyctl (a symlink), the test binary runs main, so tests can run the snap's shell scripts against the real commands
invoke(t, argv...)main_test.goRuns the real command tree in English with go-cmdr's cmdtest; its buffers make every setup command print its piped script
invokeIn(t, locale, argv...)main_test.goThe same in another locale
invokeOn(t, h, argv...), invokeWithmain_test.goWith a host of test doubles, or another command tree
check(t, errs...)main_test.goReports every failed expectation of one run
noDaemon(t)main_test.goPoints SNAP_COMMON, and with it the default stats and admin sockets, at an empty directory
unixServer(t, handler)cmd_stats_test.goServes a fake daemon on a UNIX socket

charm_test.go keeps the charms in step with what they wrap:

TestChecks
TestCharmOptionsMatchFlagsin both charms, every option is a daemon flag of the same name and default, or one of the charm's own; every flag is an option, or listed with a reason as not one
TestMachineCharmOptionsAreSnapOptionsevery machine charm option is a snap option pkgproxy-env maps (lxd-tls as lxd-tls-addr)
TestCharmServiceMatchesRockthe Kubernetes charm's Pebble service runs the rock's command (without its empty [ ] arguments), with the rock's kill-delay
TestCharmSetupScriptMatchesSetupthe setup-script action offers every tool pkgproxyctl setup sets up, in its order
TestCharmsShareCommonsrc/common.py is the same file in every charm

Expectations compare the whole output: inv.ExpectStdout, ExpectStderr, ExpectStderrTemplate, with <IGNORE> or <VARIABLE name> where output varies (host names, temporary paths, time).

Synctest rules

  • Run the test in synctest.Test; synctest.Wait() returns once every other goroutine in the bubble is blocked.
  • Never time.Sleep or poll: timeouts, TTLs and sleeps pass on the bubble's fake clock.
  • Talk HTTP over httptest.NewTestServer and its Client(): a goroutine blocked on a real socket keeps the clock from moving.

./check.sh: the Dagger checks

./check.sh is exec dagger check; GitLab CI's dagger-check job runs the same module and then dagger call coverage export --path=coverage. It needs the dagger CLI and docker or podman. The container is golang:1.27 with shared module and build caches.

CheckRuns
go-fmtgo fmt ./... prints nothing
go-vetgo vet ./...
go-lintgo tool golang.org/x/lint/golint ./...
go-buildgo build ./...
static-checkgo tool honnef.co/go/tools/cmd/staticcheck ./...
go-constgoconst, ignoring immutable, revalidate, tcp, unix, json, limit
err-checkgo tool github.com/kisielk/errcheck ./...
wslgo tool github.com/bombsimon/wsl/v5/cmd/wsl ./...
ineff-assigngo tool github.com/gordonklaus/ineffassign ./...
testgo test -race -short -coverprofile=coverage.txt -covermode=atomic ./..., then go tool cover -func
go-test-latestgo test -race -short ./... with GOTOOLCHAIN set to the latest stable Go from go.dev
snapd-testgo vet and go test -race -count=1 ./... in internal/integration/snapd
snapd-lintgofmt, staticcheck, errcheck and ineffassign in internal/integration/snapd, built with -modfile=/src/go.mod
reuse-lintreuse lint in fsfe/reuse:latest
charm-lintuv sync --locked, then uv run mypy (--strict, from the charm's pyproject.toml) in charms/pkgproxy-k8s and charms/pkgproxy
charm-testuv sync --locked, then uv run pytest -q (tests/unit) in each charm

The charm checks run in ghcr.io/astral-sh/uv:0.9.30-python3.14-trixie-slim, with a shared uv cache. --locked fails when uv.lock no longer matches pyproject.toml.

CommandDoes
dagger check -lList the checks
dagger check go-vet static-checkRun some by name
dagger check --failfastStop at the first failure
dagger call coverage export --path=coveragecoverage.txt, coverage-func.txt, Cobertura coverage.xml

The pre-commit hook

Enable once per clone (all its worktrees share it): git config core.hooksPath githooks. It needs no container and runs, with the tools go.mod pins:

CheckCommand
gofmtgofmt -l on every tracked .go file outside .dagger/
go-vetgo vet ./...
staticcheckgo tool honnef.co/go/tools/cmd/staticcheck ./...
goconston a copy of the files CI sees, with CI's ignore list
errcheckgo tool github.com/kisielk/errcheck ./...
wslgo tool github.com/bombsimon/wsl/v5/cmd/wsl ./...
ineffassigngo tool github.com/gordonklaus/ineffassign ./...
go mod tidygo mod tidy -diff in every module but .dagger, the nested ones included

It also reminds you to regenerate .dagger/dagger.gen.go when a staged .dagger Go file changed without it. Tests, golint and REUSE lint are left to ./check.sh and CI.

Commits

Every commit needs Signed-off-by:: commit with git commit -s, repair with git commit --amend -s or git rebase --signoff. CI's dco job runs dco-check on merge requests and the default branch.

The spread suite

SystemLXDWorkersSuites
ubuntu-cloud-26.04-at-lxd665every suite but lxd-upgrade, with tests/rock/ and the manual tests/charm/; the only system CI runs
ubuntu-cloud-26.04-at-lxd55.213lxd, lxd-remote, snapcraft, rockcraft, charmcraft, lxd-upgrade
ubuntu-cloud-26.04-at-barenone1the manual tests/offline-build/ alone; the project's prepare, prepare-each and restore-each do nothing on it
SuiteTasks
tests/apt/the six setup tasks, offline, store-overloaded, upstream-silent (a silent archive: apt works from the cache, served stale)
tests/snap/the six setup tasks, offline, offline-refresh, offline-session (a snapd with a serial but no session installs from the cache), store-overloaded, store-down (a store that cannot be reached at all), store-silent (a store that hangs)
tests/docker/the six setup tasks, offline
tests/go/the six setup tasks, setup-before-go, offline, offline-build (go get, build and install offline after the module cache is wiped)
tests/lxd-remote/the six setup tasks, offline, refuses-sudo
tests/lxd/the six setup tasks
tests/files/smoke, offline, volatile (SHA256SUMS online twice, then the kept copy offline, marked STALE; refused once invalidated)
tests/install/daemon, snap-option (one variant per snap option), offline-switch, auto-offline, migrate-meta, proxy-options
tests/snapcraft/, tests/rockcraft/, tests/charmcraft/pack, twice, the second from the cache
tests/lxd-upgrade/setup-again, on -at-lxd5: LXD 5.21 to 6
tests/offline-build/manual, about an hour: pack, a fresh machine with no snapd that packs tests/snapcraft/fixture once online and again offline with the leak guard and no default routes, from pkgproxy alone; pkgproxy runs outside snapd as pkgproxy-offline.service, state in /var/lib/pkgproxy-offline; what was refused is in the artifact refused.log
tests/rock/apt: downloads through the rock in docker from a named volume, and finds the cache again in a new container; options: a variable, a flag, an invalid option, and stopping it
tests/charm/manual: integration (the Kubernetes charm, on MicroK8s) and machine (the machine charm, on the VM's LXD)
TaskAsserts
setupThe setup command, piped to sh, configures the client
setup-setupTwice leaves the machine as once
undoSetup then -undo leaves the machine as it was
undo-undoA second -undo succeeds and changes nothing
undo-first-undo on a machine never set up succeeds and changes nothing
smokeThe client downloads through pkgproxy; the second download is a cache hit

tests/lib/invariants.sh runs after every task: no proxy setting left behind, offline mode never, auto-offline=off, the default routes. tests/lib/pkgproxy.sh has the pp_* helpers (pp_not for a negative, pp_user to run as ubuntu, pp_hits_delta, pp_content_re, pp_assert_no_upstream, which accepts a hit, a synthesised answer, a stale copy or a refusal, pp_assert_served_locally T [ERE], which requires hit, stale or synthesised of every access log line since T, pp_volatile_drop, pp_tool_snaps, …); pp_guard_on [all] [reject|drop] loads the leak guard, an nft rule on the daemon's cgroup: reject refuses each new connection, as an upstream that is down, drop discards the packets, as one that hangs, and all stops already open connections too. PP_UNIT, PP_COMMON, PP_STATS_SOCK and PP_ADMIN_SOCK point the helpers at a pkgproxy outside the snap; tests/lib/workload/<client>.sh has the wl_* functions the setup tasks share.

rm -f pkgproxy_*.snap && snapcraft pack     # the one snap the suite installs, in the main checkout
rm -f pkgproxy_*.rock && rockcraft pack     # the one rock tests/rock/ and tests/charm/ run
spread -list
spread garden:
spread garden:ubuntu-cloud-26.04-at-lxd6:tests/apt/
spread garden:ubuntu-cloud-26.04-at-lxd6:tests/apt/smoke
spread -debug ...                           # a shell in the VM when a task fails
spread -reuse ...                           # keep the VMs for the next run

It needs image-garden and spread (the image-garden snap has both), KVM and about 6 GiB per VM, and network access. Optionally, an infrastructure pkgproxy as PKGPROXY_UPSTREAM_URL (by default PKGPROXY_URL, which CI runners set), an address the VMs reach, never localhost: the pkgproxy under test then fetches everything through it (upstream-pkgproxy, §27), and the snapcraft, rockcraft and charmcraft suites install their tools from it with pp_tool_snaps, so that they do not warm the cache of the pkgproxy under test. tests/rock/ publishes the rock's port 3142 on 4142, beside the snap.

The charm suite

tests/charm/ is manual: spread runs it only when it is named, as the cluster and the controllers take minutes to set up.

spread garden:ubuntu-cloud-26.04-at-lxd6:tests/charm/
PKGPROXY_JUJU_CHANNEL=4/stable spread garden:ubuntu-cloud-26.04-at-lxd6:tests/charm/

Its prepare, once per VM, lets pod and container traffic through Docker's FORWARD policy (DOCKER-USER rules for cali* and lxdbr0), makes the network namespace pkgproxy-client (192.168.250.2) as a client off the cluster, and runs concierge with tests/charm/concierge.yaml: Juju from PKGPROXY_JUJU_CHANNEL (default 3.6/stable), a strictly confined MicroK8s with hostpath-storage, dns, registry and metallb (10.64.140.43 to 49), and the controllers concierge-microk8s and concierge-lxd, whose models run update-status every minute, as Juju 4 runs config-changed only after the next one. It copies the rock into localhost:32000, and takes the charms CI packed from the root of the tree, or packs them with charmcraft pack --destructive-mode.

TaskRunsAsserts
integrationcharms/pkgproxy-k8s/tests/integrationactive with the cache on storage; apt twice through the Service, the second a hit; still a hit after the pod is deleted; an option reaches the service; a refused option blocks the unit while the old configuration serves; offline mode; the stats, invalidate and setup-script actions; the LoadBalancer keeps the client's address
machinecharms/pkgproxy/tests/integrationdeployed with the snap as pkgproxy-snap; apt twice, the second a hit; an option reaches snap get; a refused option blocks the unit and keeps the old value; a refresh installs the snap again and the package is still a hit

A run takes 9 to 13 minutes, most of it concierge. pytest.xml is each task's artifact (spread -artifacts). The Kubernetes tests also run against a cluster of your own: PKGPROXY_CHARM, PKGPROXY_IMAGE, JUJU_MODEL, and for the LoadBalancer test PKGPROXY_CLIENT_NETNS and PKGPROXY_CLIENT_ADDR, then uv run pytest tests/integration in charms/pkgproxy-k8s/.

In CI

JobDoes
snap-cipacks the snap under test with snapcraft-gitlab-ci's build component, snapcraft pack --destructive-mode in a throwaway 26.04 VM, from a copy of the working tree with .git; no needs, so nothing untracked stamps the binaries +dirty; keeps it in snaps/
snap-ci-checkthe binaries are not +dirty
snap-ci-arm64, snap-ci-arm64-checkthe same, natively on an aarch64 runner; spread tests the amd64 snap only
rock-ci, rock-ci-checkpacks the rock with rockcraft-gitlab-ci into rocks/, and checks its binary
charm-ci, charm-ci-checkpacks charms/pkgproxy-k8s/ with charmcraft-gitlab-ci into pkgproxy-k8s_amd64.charm, and checks its metadata (the container's user, the cache storage) and that its virtualenv has no test packages
charm-machine-24.04, charm-machine-26.04pack charms/pkgproxy/ once for each base
spreadmoves the snap and the rock into the root of the tree and runs the -at-lxd6 tasks in five VMs, with spread-gitlab-ci's spread@0.4 component, through spread-gitlab-report; the runner's PKGPROXY_URL is the upstream of the pkgproxy under test
spread-charmruns tests/charm/ in one VM, as a matrix of PKGPROXY_JUJU_CHANNEL 3.6/stable and 4/stable
spread-offline-buildruns tests/offline-build/pack with the snap snap-ci packed, in a VM of its own: on a schedule of main that sets PKGPROXY_OFFLINE_BUILD=1, which the publishing jobs skip, and by hand in any other pipeline; retries: 1
rockafter spread, pushes the rock, edge from main and the version from a tag, and names it by digest as ROCK_IMAGE
snap-edge, snap-stable, charm-edge, charm-stablepublish to the Snap Store and Charmhub

A retry of a spread job, GitLab's or by hand, runs only the tasks that did not pass: the component keeps them in .spread-rerun/, cached under spread-rerun-$CI_PIPELINE_ID-$CI_JOB_NAME_SLUG.

spread-charm takes 9 to 13 minutes of a large KVM slot, so it runs only on a merge request labelled spread-charm (add the label, then push or run the pipeline again), on a pipeline with PKGPROXY_SPREAD_CHARM=1 (by hand or on a schedule, which then runs snap-ci, rock-ci, charm-ci and spread-charm alone), and always on main and version tags. Ask for it on every merge request that changes the charms, the rock or tests/charm/.

Roads from here

SETUP AND THE COMMAND LINE

OPERATIONS

INSIDE PKGPROXY · for contributors