pkgproxy

pkgproxy keeps a local copy of the packages and images your machine downloads, so the next download comes from disk instead of the internet. It works with:

• apt (Debian and Ubuntu packages)
• snapd (snaps)
• Go modules
• Docker images from Docker Hub
• LXD images, including the ones snapcraft, rockcraft and charmcraft builds use

Files that never change, like .deb packages or Docker image layers, are kept forever. Lists that do change, like apt’s package indexes, are checked against the original server every few minutes. Every package, module and image is checked against its published hash before pkgproxy stores it.

Install

sudo snap install pkgproxy

The service starts right away and listens on port 3142, and on port 3143 for LXD. No interface needs connecting: pkgproxyctl setup lxd and pkgproxyctl setup snapcraft print scripts that ask LXD for its network themselves.

Set up a client on the same machine

Run the command for each tool you want to use pkgproxy:

If your shell cannot find pkgproxyctl, use pkgproxy.pkgproxyctl. Run a command without | sh to see what it will do and why. Add -undo to get the commands that remove the setup again.

A few notes:

• apt: keep your sources.list as it is. Only http:// sources are cached; https:// sources still work but are not cached.

• Docker: if pkgproxy is down, Docker downloads from Docker Hub directly. Only public images are cached.

• LXD image remotes: run this as yourself, not with sudo. It adds the remotes pkgproxy-images, pkgproxy-ubuntu and pkgproxy-ubuntu-daily. Use them in place of images:, ubuntu: and ubuntu-daily:, for example lxc launch pkgproxy-images:alpine/edge. It also adds the two remotes snapcraft, rockcraft and charmcraft use for their build images, so they pick them up with no further changes.

• LXD containers: this makes apt, snapd and Go inside your containers use pkgproxy. Attach it with --profile pkgproxy, or use -project to apply it to every container in a project. Run it, and setup snapcraft, as yourself: neither needs root, but both need you to be in the lxd group, as lxc does.

• snapcraft, rockcraft and charmcraft: run two commands, one for the build images and one for what the build downloads:

  pkgproxyctl setup lxd-remote | sh
  pkgproxyctl setup snapcraft | sh

  For rockcraft or charmcraft, run pkgproxyctl setup rockcraft or setup charmcraft in place of setup snapcraft. The second command deletes the tool’s cached base instance, so the next build takes longer while it is created again with pkgproxy in place. A charm’s Python packages, which charmcraft fetches from PyPI over HTTPS, and its charm libraries from Charmhub do not go through pkgproxy.

Set up a client on another machine

Machines without pkgproxy installed can use it too. Open http://<host>:3142/ in a browser, or run curl http://<host>:3142/, and run the command shown for each tool, for example:

curl -fsSL 'http://<host>:3142/setup/apt' | sudo sh

The page also shows the undo command and the plain commands, for machines without curl. It offers apt, snapd, Go, Docker and LXD image remotes; the LXD containers and snapcraft setups only work on the machine that runs pkgproxy. To hide the page, run sudo snap set pkgproxy setup-page=false.

On the machine that runs pkgproxy, the pkgproxy entry in the applications menu opens the page in your browser, and so does pkgproxy.open in a terminal. If the entry seems to do nothing, run pkgproxy.open to see why. There the page also says how large the cache is and how much space is left on its disk, and links to a page, http://localhost:3142/pkgproxy/cache, with how much downloading the cache has saved and a map of what is cached. Visitors from other machines see neither.

Check that it works

pkgproxyctl stats

shows what is in the cache and how much downloading it saved, for every tool together. Run it after an apt update or a snap refresh; if the numbers grow, the tool is using pkgproxy. Add -r 1h to see only the last hour.

If an apt mirror served a broken package list and pkgproxy kept it, throw the copy away with pkgproxyctl cache invalidate <url>, giving the http:// address apt downloads it from. Lists refresh on their own every few minutes anyway. This works for apt only; see DEPLOYMENT.md for snaps, Go modules, Docker and LXD images.

If a tool fails, or the numbers do not grow, see docs/troubleshooting.md: where the log is, what the errors mean, and how to tell whether a tool uses pkgproxy. pkgproxyctl version shows which version you run.

More

• The documentation online: everything below, as web pages.
• DEPLOYMENT.md: where the cache lives, how to free disk space (pkgproxy never deletes anything on its own), working offline (on Linux pkgproxy also goes offline with the machine’s network), ports, timeouts and other settings.
• docs/troubleshooting.md: the log, the errors clients see, and what to do about them.
• NEWS: what changed in each release.
• docs/statsapi.md: the statistics API behind pkgproxyctl stats, for your own dashboards.
• docs/design.md: how pkgproxy works inside.
• Issues: report a bug or ask for a feature.

License

pkgproxy is licensed under the Apache License 2.0.