pkgproxy 0.11 release notes

pkgproxy 0.11 is out. If you haven’t met it yet: pkgproxy is the caching proxy I run at home for apt, snaps, Go modules, LXD images and Docker Hub, so that the twentieth test VM of the day doesn’t download the same packages the other nineteen already did.

This one comes with a small chore, so let me get that out of the way first.

Do this after the refresh

If you ever ran pkgproxyctl setup lxd, or setup snapcraft, rockcraft or charmcraft, you have a little work to do. If you only use pkgproxy for apt, snaps, Go, Docker or LXD images, you can skip to the next section.

The snap refreshes on its own, but you can grab 0.11 right now and ask pkgproxy what it thinks of your machine:

sudo snap refresh pkgproxy
pkgproxyctl setup status

Every row that says needs setup again wants its setup command run once more, as yourself, the same way you ran it the first time. For me that was:

pkgproxyctl setup lxd | sh
pkgproxyctl setup snapcraft | sh

Then restart the containers that use the profile. When setup status is happy with everything, you can throw away the files the old profile used:

sudo rm -r /var/snap/pkgproxy/common/lxd

That’s it. Now for what you just did, and why.

What changed under the hood

Until now, the LXD profile that pkgproxy set up for you mounted three files that pkgproxy wrote under /var/snap/pkgproxy/common/lxd: a bit of apt configuration, a bit of snapd configuration, and a tiny script that switches Ubuntu’s archives from https to http, so that pkgproxy can actually cache them. It worked, but it had a catch I’ll get to in a moment.

In 0.11 the profile carries all of that itself. It has two small hooks in its raw.lxc: one writes the apt and snapd configuration into a container right before it starts, and the other takes it out again once the container stops. A container only has those files while it runs with the profile, so taking the profile away leaves nothing behind.

So, step by step:

If you’d like to see the hooks before you trust them with your containers (I would), the setup script prints both of them, decoded, as comments. A hook runs as root on the host, so they’re careful: they write nothing on the host itself, they leave a container without apt or systemd alone, and they refuse to follow a symbolic link that something inside the container could have planted to point back at the host.

The point of all this: LXD on other machines

Here’s the catch. The old profile only worked on the machine pkgproxy ran on, because that’s the only place those files existed. I have pkgproxy on one machine and LXD on several, so that bugged me.

Now that the profile carries everything with it, the setup page pkgproxy serves offers LXD containers and the three craft tools too. On any machine with LXD, as yourself, in the lxd group:

curl -fsSL http://pkgproxy.lan:3142/setup/lxd | sh
curl -fsSL http://pkgproxy.lan:3142/setup/snapcraft | sh

Containers talk to pkgproxy at the address you fetched the script from. The script looks the name up on your machine, because containers don’t necessarily know it. On the machine pkgproxy itself runs on, they use the LXD bridge, as before.

Setup scripts you can run anywhere

The other thing I wanted for my test VMs is one set of commands I can throw at every machine, whatever it has installed. Each setup script now does nothing on a machine without its tool, and says so: the apt one on Fedora, the Docker one without Docker, the LXD ones without LXD.

And when a script can’t do its job, it now says so too. setup apt | sh run without root used to quietly do nothing and look happy about it; now it tells you to run it as root, and fails.

Friendlier messages

I spent a good part of this cycle reading what pkgproxyctl actually says when things go wrong, and a lot of it wasn’t written with the reader in mind. Here are a few before-and-after pairs.

If pkgproxy isn’t running, pkgproxyctl used to dump the HTTP request it was trying to make:

pkgproxyctl error: cannot reach pkgproxy's admin socket (/run/pkgproxy/admin.sock): Delete "http://pkgproxy-admin/pkgproxy/v1/cache": dial unix /run/pkgproxy/admin.sock: connect: no such file or directory

Now it says what’s going on first:

pkgproxyctl error: cannot invalidate a cache entry: pkgproxy is not running: cannot reach pkgproxy's admin socket at /run/pkgproxy/admin.sock: connect: no such file or directory

When an upstream misbehaves, pkgproxy leaves it alone for a while and serves what it has from the cache. pkgproxyctl offline status used to put that in circuit-breaker terms:

upstream api.snapcraft.io: open, next probe in 30 s

which, quite fairly, made people think the upstream was open and everything was fine. Now it reads:

upstream api.snapcraft.io: left alone after trouble, served from the cache; next attempt in 30s

pkgproxyctl setup used to accept any address you gave it, so setup apt -addr nonsense happily pointed apt at http://nonsense. Now it checks the address the way pkgproxy itself does:

pkgproxyctl error: cannot use -addr "nonsense": it must be host:port or :port, such as ":3142" or "127.0.0.1:3142"

With the snap, you set options with snap set, which hands them to pkgproxy as environment variables. A bad one used to be reported under the name of a command-line option you never typed, with an example for the wrong port:

pkgproxy error: cannot use -lxd-tls-addr "nonsense": it must be host:port or :port, such as ":3142" or "127.0.0.1:3142"

Now it names the variable it actually came from, and its own port:

pkgproxy error: cannot use PKGPROXY_LXD_TLS_ADDR "nonsense": it must be host:port or :port, such as ":3143" or "127.0.0.1:3143"

A few more small ones: a mistyped command is named in the error (unknown command "bogus", where it used to say just unknown command), a bad option value is reported once instead of twice, and pkgproxy no longer logs a pointless warning for every suite on every apt update.

The full list is in the release notes. If something looks off, the troubleshooting page is a good start, and the issue tracker is where to tell me about it.

← All posts