The sides' packages
contributors for contributors
How each side (proxy, snapproxy, gomodproxy, registryproxy, lxdproxy, filesproxy) is built from a store, the upstream transport and shared helpers, and what the Go and Docker sides do inside.
On the map: inside pkgproxy
01 What this topic covers
How each side is built from a store, a transport and shared helpers, and what the Go and Docker sides do inside.
A side is the part of pkgproxy that serves one kind of client: apt, snap, Go, Docker, LXD or files by checksum. If you have not seen A request in code, start there. You will see how the daemon command builds every side from a store, a transport and shared helpers, and the revalidate package that three of them share. Then you look inside the Go side and inside the Docker side.
02 How a side is built
newAptHandler … newFilesHandler in
cmd/pkgproxy, each with fslocal.New(cacheDir/<side>).
A disabled side's constructor returns nil, and the gateway field stays a
nil interface. Each handler names the object a request is for with
reqlog.SetObject, for the cache page's live views.
Each side is a Handler type in its own package, and the daemon command builds them one after another before it listens. Every side gets a local store of its own, rooted at a subdirectory of the cache directory named after it, with the logger and the statistics recorder. Four fetch through the shared upstream transport; apt and the files side wrap the same layers around a transport of their own. Docker and LXD can be turned off, and are then left out of the gateway. Inside, the sides share helpers: fills for large blobs, flight for one shared fetch, revalidate for documents that change.
03 The revalidate package
revalidate.Group.Do shares one detached fetch per key, bounded
by Policy.Wait (10 s) and Policy.RetryInterval
(1 min). The snap and Docker sides serve stale entries with their own code.
In the code, the apt, Go and LXD handlers share the revalidate package. A Group runs one revalidation per key, detached from the request and joined by every request for that key. Do waits for it up to the policy's Wait, ten seconds by default, then returns ErrSlow and records the host as failing. Within the RetryInterval, one minute, Do returns ErrFailing at once, without fetching. Each failure carries the offline switch's epoch, so a change of mode forgets it. The apt handler names the two fields RevalidateWait and OriginRetryInterval.
04 Inside the Go side
gomod.Request names the endpoint; Endpoint.Immutable
picks the path. fetchAndStoreImmutable runs under
flight, hashes with dirhash and commits after
verifyAgainstSumdb; serveTTL uses
revalidate.ServeDoc. See design §16.
The gateway hands every path under the go base path to the Go handler. The gomod package parses the request into a module, a version and one of the five endpoints. A go.mod or zip miss is one shared fetch, detached from the client that started it: the body goes to the store and, for a zip, to a temporary file, and is hashed with dirhash. The entry is committed only after the checksum database check, then served from the store. Unlike apt's pool files, it is not streamed while it downloads. The three changing endpoints go through the shared revalidation code.
05 Inside the Docker side
tags/<name>/_/<tag>/<id> holds a digest;
manifests/sha256/… and blobs/sha256/… hold
immutable content. revalidateTag uses headTag;
a stale pointer is served when the registry cannot be asked.
The registry proxy answers the /v2 version check itself, so Docker sends no credentials, and fetches with an anonymous pull token of its own. Everything in its store is addressed by digest. Manifests and blobs are stored once per digest, whatever repository they came through, and are checked against that digest on the way in; a blob streams through a fill. Only a tag can change. It is stored as a small pointer to a digest, one per set of accepted media types, and after five minutes it is checked with a HEAD request, which Docker Hub does not count as a pull.
06 Where to go next general version
Next: internal/fill, for the fills blob sides stream through; Store and keys, for the store every side writes; Go module proxy, for the Go side from outside.
You now know what every side is built from, and what the Go and Docker sides add for their own clients. To see the fills that the sides with large blobs stream through, watch internal/fill. To see the store they all write to, watch Store and keys. To see the Go side as a developer uses it, watch Go module proxy.
Read more
Each side as the daemon builds it
RunFlag calls the constructors in this order, after opening the stats recorder, and before it listens. Each opens fslocal.New(-cache-dir/<dir>) and passes the logger, stats.Recorder and the offline switch.
| Side | Package and type | Constructor | Store dir | Gateway field |
|---|---|---|---|---|
| apt | proxy.Handler | newAptHandler | apt/ | Apt |
| snap | snapproxy.Handler | newSnapHandler | snap/ | Snap, inside offline.MiddlewareAnswer |
| Go | gomodproxy.Handler | newGoHandler | go/ | Go, at -go-base-path |
| LXD | lxdproxy.Handler | newLxdHandler | lxd/ | Lxd, at -lxd-base-path; nil when off |
| Docker | registryproxy.Handler | newDockerHandler | docker/ | Docker; nil when off |
| files | filesproxy.Handler | newFilesHandler | files/ | Files, at /files |
Upstream settings
| Side | Upstream | Flag and default | Other settings |
|---|---|---|---|
| apt | whatever host the client's URL names | none | -apt-revalidate-ttl, -min-forced-revalidate-interval, -fill-drain-timeout |
| snap | the Snap Store API, and the CDN URLs it hands out | -snap-store-url https://api.snapcraft.io | -snap-external-url, -snap-revalidate-ttl, -fill-drain-timeout |
| Go | a module proxy | -go-upstream https://proxy.golang.org | -go-sumdb sum.golang.org (or off), -go-base-path /go, -go-revalidate-ttl |
| Docker | one registry | -docker-upstream https://registry-1.docker.io (off turns the side off) | -docker-revalidate-ttl |
| LXD | named simplestreams servers | -lxd-remote name=url, repeatable (off turns the side off) | -lxd-base-path /lxd, -lxd-revalidate-ttl, -lxd-checkpoint-interval, -lxd-tls-addr, -fill-drain-timeout |
| files | whatever URL the client names | none | -files-allow-host, -files-allow-network, -files-max-size (8G) |
Every -<side>-revalidate-ttl defaults to -revalidate-ttl, 5 minutes. The default LXD remotes: pkgproxy-images (images.lxd.canonical.com), pkgproxy-ubuntu and pkgproxy-ubuntu-daily (cloud-images.ubuntu.com /releases, /daily), and craft-providers' craft-com.ubuntu.cloud-buildd and craft-com.ubuntu.cloud-buildd-daily (/buildd/releases, /buildd/daily).
What each side caches
| Side | Kept forever (immutable) | Revalidated after its TTL | Not cached |
|---|---|---|---|
| apt | pool/**, dists/**/by-hash/**, checked against the hashes the parsed indexes list | other dists/**: Release, InRelease, Packages, Sources | URLs without dists or pool; CONNECT tunnels |
| snap | .snap files by sha3-384 | info, find, catalogue answers and assertions | answers to a user's macaroon that show private snaps; anything on a brand store's device session |
| Go | .mod and .zip of a version, checked against the checksum database before commit (unchecked with -go-sumdb off) | @v/list, @latest, .info | a path that is not one of the protocol's five endpoints |
| Docker | blobs and manifests by sha256 digest, checked against it | tags: a pointer to a digest per set of accepted media types, checked with a HEAD | methods other than GET and HEAD (405); tag lists and the catalog (registry 404) |
| LXD | image files by the sha256 their product catalog lists | streams/v1/index.json and product catalogs | a path no parsed catalog listed: 404, never fetched |
| files | the file by sha256 or sha512, whatever its URL | nothing | a private address outside -files-allow-network; a host outside -files-allow-host; a file over -files-max-size |
Shared helpers by side
| Side | fill.Group (large blobs, streamed) | flight.Group (one shared fetch) | revalidate.Group (bounded revalidation) | Stale entries on upstream failure |
|---|---|---|---|---|
| apt | pool files | dists/**, and the passthrough fallback | yes, RevalidateWait, OriginRetryInterval | yes |
| snap | .snap files | relayed metadata fetches, and assertions fetched ahead | no | its own code |
| Go | no | .mod and .zip | yes, through revalidate.ServeDoc | yes |
| Docker | blobs | manifests | no | its own code: a stale tag pointer |
| LXD | image files | no | yes, through revalidate.ServeDoc | yes |
| files | files | no | no | not applicable |
revalidate.Policy: Wait defaults to revalidate.DefaultWait (10s), after which Do returns ErrSlow; RetryInterval to DefaultRetryInterval (1m), within which a host that failed gets ErrFailing without a fetch; Epoch is the offline switch's, so a change of mode forgets failures.
Disabled sides
newDockerHandler returns nil for -docker-upstream off, newLxdHandler nil for -lxd-remote off. RunFlag assigns them to the gateway only when non-nil: a nil *lxdproxy.Handler stored in an http.Handler field would be a non-nil interface, and the gateway's h.Lxd != nil check would call through it. With Docker off, a registry-shaped path still matches and gets 404; with LXD off, /lxd falls through to the snap prefixes and then 404.
In the documentation
Roads from here
GO MODULES
- Go module proxy — the Go side from the outside
DOCKER
- Docker Hub registry mirror — the Docker side from the outside
INSIDE PKGPROXY · for contributors
- A request in code — how a request reaches a side
- internal/fill — the fills blob sides stream through
- Store and keys — the store every side writes
- The source tree — the rest of the source tree