On Kubernetes

running pkgproxy for administrators for CI operators

The pkgproxy-k8s charm runs the pkgproxy rock on Kubernetes with Juju, one unit with its cache on Juju storage, options that are the daemon's settings checked before they apply, a ClusterIP or LoadBalancer Service, and actions for stats, invalidating a file and setup scripts.

On the map: running pkgproxy

▶ Watch this deck — 7 scenes, ~4 min

Part of journeys

01 What this topic covers

ABOUT THIS TOPIC On Kubernetes the pkgproxy-k8s charm runs the rock with Juju 1 The rock, deployed by Juju 2 The cache on Juju storage 3 Options are the daemon's settings 4 A Service for the clients 5 Actions instead of a shell
The topic title, On Kubernetes, and its 5 scenes in order: The rock, deployed by Juju; The cache on Juju storage; Options are the daemon's settings; A Service for the clients; Actions instead of a shell

The pkgproxy-k8s charm: the rock as its workload, the cache on Juju storage, options, Services and actions.

The Kubernetes charm runs pkgproxy on Kubernetes, with Juju, as the shared cache of the machines around a model. You will see the rock it deploys and how a release pairs the two. Then come the cache on Juju storage, options that are the daemon's settings, and the Service clients reach it on. Last come the actions that stand in for a shell.

02 The rock, deployed by Juju

THE KUBERNETES CHARM $ juju deploy pkgproxy-k8s a pod the charm's container workload: the rock pkgproxy under Pebble a Charmhub release charm revision + pkgproxy-image the rock of the same commit, by digest $ juju deploy ./CHARM --resource pkgproxy-image=IMAGE a charm you packed: name its rock one unit a second unit: blocked
juju deploy pkgproxy-k8s makes a pod with the charm's container and the workload container running the rock; Charmhub releases each charm revision with the rock of the same commit, by digest; a local charm names its rock with --resource; a second unit is blocked

juju deploy pkgproxy-k8s: Charmhub releases the charm with the pkgproxy-image resource of its commit, by digest. A local charm needs --resource pkgproxy-image=IMAGE. One unit only.

The Kubernetes charm runs pkgproxy in a pod, with Juju. Its workload is the rock, the same image you can run with docker, given to the charm as a resource. Each release on Charmhub pairs a charm revision with the rock built from the same commit, named by its digest, so a channel always runs the rock it was tested with. A charm you pack yourself has no rock of its own: name one when you deploy it. The charm runs one unit, and a second one leaves it blocked.

03 The cache on Juju storage

STORAGE $ juju deploy pkgproxy-k8s --storage cache=100G cache at least 16G /var/cache/pkgproxy pod a persistent volume claim the next pod cache, stats, LXD certificate kept the charm gives the volume to 584792, the rock's user no extended attributes: blocked
The pod mounts the cache storage, a persistent volume claim of at least 16G, at /var/cache/pkgproxy; when the pod is replaced, the next pod mounts the same claim, with the cache, stats and LXD certificate; the charm gives the volume to 584792; a volume without extended attributes blocks the unit

Storage cache: a filesystem of at least 16G at /var/cache/pkgproxy, owned by 584792, the rock's user. Size it with --storage cache=SIZE. Without extended attributes the unit is blocked.

The cache is Juju storage named cache, a file system of at least sixteen gigabytes, mounted where the rock keeps its cache. On Kubernetes it is a persistent volume claim, so it outlives the pod. After a restart or a juju refresh, the cache, the statistics and the LXD certificate are still there. pkgproxy runs as the rock's own user, and the charm gives that user the volume before pkgproxy starts. A volume that cannot keep extended attributes leaves the unit blocked, with a message that says why.

04 Options are the daemon's settings

OPTIONS $ juju config pkgproxy-k8s offline-mode=never PKGPROXY_OFFLINE_MODE=never set only when it is not the default pkgproxy -check-config, in the workload accepted: applied pkgproxy restarts with it refused: unit blocked the daemon's message; last good kept the model's proxy settings juju-http-proxy, juju-https-proxy, juju-no-proxy → HTTP_PROXY, HTTPS_PROXY, NO_PROXY
juju config sets an option; the charm turns it into a PKGPROXY_ variable when it is not the default, runs pkgproxy -check-config in the workload, and either applies it or blocks the unit with the daemon's message while pkgproxy keeps the last good settings; the model's proxy settings become HTTP_PROXY, HTTPS_PROXY and NO_PROXY

juju config pkgproxy-k8s NAME=VALUE becomes PKGPROXY_NAME when it is not the default. pkgproxy -check-config runs first; a refusal blocks the unit. The model's juju-http-proxy and the others become HTTP_PROXY, HTTPS_PROXY and NO_PROXY.

Each charm option is the daemon setting of the same name. The charm passes it as its environment variable only when it differs from the default, so an option set back to its default leaves the environment. Before it applies anything, the charm runs pkgproxy with the check config flag inside the workload. A refused value blocks the unit with the daemon's own message, and pkgproxy keeps running with the last good settings. The model's proxy settings reach the daemon as its proxy variables.

05 A Service for the clients

SERVICES pkgproxy 3142 · 3143 a pod in the cluster pkgproxy-k8s.MODEL.svc:3142 a machine outside no address $ juju config pkgproxy-k8s service-type=loadbalancer APP-lb a LoadBalancer Service keeps each client's own address for the access log and trusted-networks refused by the cluster: blocked, with juju trust APP --scope=cluster
Clients in the cluster use the application's ClusterIP Service on ports 3142 and 3143; with service-type=loadbalancer a second Service, APP-lb, gives machines outside the cluster an address and keeps their own address for the access log and trusted-networks; where the cluster refuses, the status names juju trust

service-type=cluster (default): pkgproxy-k8s.MODEL.svc:3142. service-type=loadbalancer adds APP-lb with externalTrafficPolicy: Local. Refused: juju trust APP --scope=cluster.

Clients in the cluster reach pkgproxy on the application's Service, by its name in the model, on port 3142, and 3143 for LXD. Machines outside the cluster need an address of their own. Setting the service-type option to loadbalancer adds a second Service, of type LoadBalancer, that keeps each client's own address, for the access log and the trusted-networks option. Juju 3.6 lets the charm make that Service. Where the cluster refuses, the unit is blocked, and its status names the juju trust command to run.

06 Actions instead of a shell

ACTIONS the rock has no shell; pkgproxyctl runs as an action $ juju run pkgproxy-k8s/0 stats $ juju run pkgproxy-k8s/0 invalidate url=http://.../dists/noble/InRelease $ juju run pkgproxy-k8s/0 setup-script client=apt a script to run on the client machine the address in the script, the first that is set public-url the load balancer's address APP.MODEL.svc:3142 offline mode is not an action: juju config pkgproxy-k8s offline-mode=offline
Three juju run commands: stats, invalidate with a URL, and setup-script with a client; the setup script names public-url, else the load balancer's address, else the Service in the cluster; offline mode is an option, not an action

juju run pkgproxy-k8s/0 stats, invalidate url=URL and setup-script client=CLIENT [undo=true], for apt, snap, go, docker, lxd, lxd-remote, snapcraft, rockcraft and charmcraft. Offline mode is the offline-mode option.

The rock has no shell, so the commands of pkgproxyctl are actions instead, run with juju run. Stats prints the report. Invalidate throws away the cached copy of one apt file, given by its URL. The setup-script action prints the script that points one tool on a client machine at this pkgproxy. It names the public-url option when it is set, else the load balancer's address, else the Service in the cluster. There is no action for offline mode: it is an option, so juju config always says what pkgproxy does.

07 Where to go next

WHERE NEXT one unit, a cache that outlives its pods you are here On Kubernetes In a container the image the charm runs Juju machines the snap on a machine, with Juju The setup page pointing client machines at it
Where next from On Kubernetes: In a container, the image the charm runs; Juju machines, the snap on a machine; The setup page, pointing client machines at it

Next: In a container, for the rock; Juju machines, for the machine charm; The setup page, for pointing clients at it.

pkgproxy now runs on Kubernetes as one unit, with its cache on a claim that outlives its pods, and clients inside and outside the cluster can reach it. To see the image the charm runs, watch In a container. To run the snap on a machine with Juju instead, watch Juju machines. To point client machines at it, watch The setup page.

Read more

Deploying

CommandWhat it does
juju deploy pkgproxy-k8sthe charm from Charmhub, with the rock released with it
juju deploy pkgproxy-k8s --storage cache=100Gthe same, with a cache larger than the 16G minimum
juju deploy ./CHARM --resource pkgproxy-image=IMAGEa charm you packed, with the rock you name
  • Charm pkgproxy-k8s, base Ubuntu 26.04, amd64. It needs Juju 3.6 or later.
  • Channels on Charmhub: latest/edge follows the main branch, latest/stable the releases. Each charm revision is released with the rock of the same commit, by digest.
  • One unit. A second one leaves the application blocked.

Options

Each option is the daemon setting of the same name, passed as its PKGPROXY_* variable only when it differs from the default. Settings has what each one does.

OptionDefaultSets
public-urlemptythe address the setup page tells visitors to use
setup-pagetruethe setup page at / and its scripts under /setup/
trusted-networksemptynetworks whose visitors may see the cache page
offline-modestore-overloadedstore-overloaded, never or offline
revalidate-ttl5m0show long lists are served before they are checked
upstream-idle-timeout1m0show long an upstream may send nothing
upstream-retry-for2m0show long a broken-off download is asked for again
docker-upstreamhttps://registry-1.docker.iothe Docker Hub registry, or off
files-allow-hostemptyhosts files by checksum may come from
files-allow-networkemptyprivate networks files may come from
files-max-size8Gthe largest file by checksum
service-typeclustercluster, or loadbalancer for a Service with an outside address (the charm's own)
lxd-tlstruefalse turns port 3143 off (lxd-tls-addr=off)
  • Before applying a change, the charm runs pkgproxy -check-config in the workload with the new environment. A refused value blocks the unit with the daemon's message, and pkgproxy keeps the last good settings.
  • The model's juju-http-proxy, juju-https-proxy and juju-no-proxy reach pkgproxy as HTTP_PROXY, HTTPS_PROXY and NO_PROXY, in both spellings.
  • A change that is accepted restarts pkgproxy, in seconds.

Storage and ports

Storage cachea filesystem of at least 16G, mounted at /var/cache/pkgproxy; a persistent volume claim that outlives the pod
OwnerUID and GID 584792, the rock's user; the charm hands the volume to it before pkgproxy starts
Extended attributesrequired; a volume without them blocks the unit
Ports3142, and 3143 unless lxd-tls is false
In the clusterhttp://pkgproxy-k8s.MODEL.svc:3142
service-type=loadbalancera Service APP-lb of type LoadBalancer, with externalTrafficPolicy: Local, so pkgproxy sees each client's own address

Where the cluster refuses to let the charm manage APP-lb, the unit is blocked with service-type=loadbalancer needs trust: juju trust APP --scope=cluster, and pkgproxy keeps serving the clients in the cluster. Back on cluster, and when the application is removed, APP-lb is deleted.

Actions

ActionParametersResult
statsnoneoutput: the report of pkgproxyctl stats
invalidateurl (required): the http:// address of an apt file under dists/ or pool/output: what pkgproxyctl cache invalidate printed
setup-scriptclient (required): apt, snap, go, docker, lxd, lxd-remote, snapcraft, rockcraft or charmcraft; undo (default false)script: the script to run on the client
  • Run them with juju run pkgproxy-k8s/0 ACTION [PARAM=VALUE ...]. A failure gives what pkgproxyctl printed.
  • setup-script uses the host and port of public-url when it is set (port 80 when it names none), else the LoadBalancer Service's address, else APP.MODEL.svc:3142. For lxd-remote it adds port 3143 on the same host.
  • setup-script refuses docker when docker-upstream is off, and lxd-remote when lxd-tls is false.
  • Run the scripts for apt, snap and docker as root, for go as yourself, and for lxd, lxd-remote and the craft tools as the user who runs lxc. Each craft tool takes its base images through lxd-remote, so set that up too.
  • There is no action for offline mode: set the offline-mode option.

Status

The first that applies:

StatusMessage
blockedscale to one unit: more are not supported yet (pkgproxy#111)
blockedan unknown service-type, or the juju trust message above
blockedthe cache storage does not keep extended attributes: use another storage pool
blockedthe daemon's message about a refused option
waitingwaiting for the cache storage, waiting for Pebble, waiting for pkgproxy to answer
activeempty; offline: serving the cache only; or offline by itself: REASON

Offline is active, not blocked: what the cache holds is still served.

In the documentation

Roads from here

CORE · the path every side shares

SETUP AND THE COMMAND LINE

RUNNING PKGPROXY

OPERATIONS