Immutable or revalidated
core for everyone
Which cached entries are kept forever and which are rechecked after a short TTL on every side, how a recheck is made, what a client asking for a fresh copy changes, and what is served when the upstream fails.
On the map: core
01 What this topic covers
Which entries are kept forever and which are rechecked after a short window, on every side.
Some files never change once published, and pkgproxy keeps them forever; lists that change are checked again with the upstream after a short time. It helps to have seen A request first. Here you see the two kinds of entry on every side, the short window and the recheck that follows it, and how each side can have its own window. Then what changes when apt asks for a fresh copy, and what is served when the recheck fails.
02 Two kinds of entry on every side
Content addressed by hash or by version is immutable: fetched once, verified and cached forever. Indexes, version lists and tags are mutable and revalidated after a TTL. Files by checksum have no mutable part.
Every side of pkgproxy sorts what it caches into two kinds. Content named by its hash or by a fixed version never changes: an apt package in the pool, a snap file, a Go module's zip, a Docker layer or manifest fetched by digest, an LXD image file, and every file fetched by checksum. These are kept forever and never checked again. Lists that move are rechecked after a short time: apt's Release and Packages indexes, snap information, Go version lists, Docker tags, and LXD's image lists.
03 A short TTL, then a recheck
Inside the TTL a mutable entry is a hit. After it, apt sends
If-Modified-Since: 304 renews the entry,
200 replaces it. The other sides fetch again; Docker resolves
a tag with HEAD.
A list that moves is stored with the time it was fetched. For five minutes by default, every request for it is a hit, answered from disk, so a fleet running apt update together costs the upstream one check. After that, apt's indexes are rechecked with a conditional request carrying the cached copy's Last-Modified date. A 304 keeps the copy and starts its five minutes again; a 200 replaces it. Go and LXD lists and snap information are fetched again in full, and a Docker tag is resolved again with a HEAD request.
04 One window, overridden per side for administrators for CI operators
-revalidate-ttl (5m) applies to every side;
-apt-revalidate-ttl, -snap-revalidate-ttl,
-go-revalidate-ttl, -lxd-revalidate-ttl and
-docker-revalidate-ttl override it for one side each, with
snap options of the same names. Zero is refused. A snap store
max-age takes precedence; offline mode makes every window
endless.
One setting, the -revalidate-ttl, sets the window for every side, five minutes by default. Each side can override it with its own flag and snap option: the apt, snap, go, LXD and docker revalidate TTLs. A side whose own setting is not set takes the global value. A window must be longer than zero, and pkgproxy refuses zero before it starts. The snap side also keeps an answer for the max age the Snap Store announces with it, and uses its TTL for answers that announce none. A longer window means fewer requests upstream and older lists. Offline mode freezes every window.
05 When apt asks for a fresh copy
Cache-Control: no-cache, max-age=N or
Pragma: no-cache force a conditional GET, except within
-min-forced-revalidate-interval (30s) of the last
check. A 304 keeps the entry; pkgproxyctl cache
invalidate removes one.
Apt update asks for a fresh copy of each index: it sends Cache Control, max age zero. pkgproxy treats that as a reason to check, not to download again: an index inside its five minutes gets a conditional request upstream before it is served. To bound the traffic clients can cause, an index checked within the last thirty seconds is served from disk anyway, and forced checks of one file share one request. A check only confirms what is cached. To throw a cached apt file away, use pkgproxyctl cache invalidate.
06 When the recheck fails
A failed revalidation serves the stale entry with a warning. apt, Go and
LXD wait at most 10 s for it and skip a failing host for 1 min. For apt, an origin
404/410 is relayed as 404; a hash
mismatch is passed through uncached.
When a recheck fails, because the mirror is unreachable, stalls, or answers with a 5xx, the cached copy is served anyway and a warning is logged, so apt update keeps working while the mirror is down. For apt, Go and LXD, a request waits at most ten seconds for the recheck, then gets the cached copy while the recheck carries on, and a host that failed is left alone for a minute. Two answers are not covered up: a 404 or 410 from the mirror reaches apt as a 404, and content that fails its hash check is not cached.
07 Where to go next general version
Next: Offline mode, for freshness frozen; pool and dists, for apt in detail; Stalled upstreams, for when an upstream counts as stalled.
Now you know why a package is kept forever while an index is checked again after a few minutes, and why an upstream that is down still leaves you the cached copy. To see what happens when nothing may be checked again, watch Offline mode. To see apt's pool and dists in detail, watch pool and dists. To see when an upstream counts as stalled, watch Stalled upstreams.
Read more
Every side's entries
Kept forever means fetched once, checked against its hash where one is known, and never asked about again. Rechecked means served from the cache for the window in the last column, then checked with the upstream. Relayed means passed between client and upstream without being kept.
apt
| Entry | Policy | Window |
|---|---|---|
anything under pool/, and dists/…/by-hash/… | kept forever, hash checked | none |
Release, InRelease, Release.gpg | rechecked with If-Modified-Since | apt-revalidate-ttl |
every other file under dists/ (Packages, Sources, Contents, Translation, diff indexes) | rechecked with If-Modified-Since | apt-revalidate-ttl |
a URL with no pool or dists path segment | relayed | none |
an https:// source, through a CONNECT tunnel | relayed, unseen | none |
An index with no Last-Modified from the mirror, or one a newer index already shows to be superseded, is fetched again in full instead of conditionally.
Snap Store
| Entry | Policy | Window |
|---|---|---|
| snap files, by their SHA3-384 | kept forever | none |
snap-revision assertions, and account, snap-declaration, store or validation-set assertions pinned to a sequence number | kept forever | none |
every other assertion, account-key included | rechecked | snap-revalidate-ttl |
| snap info, find, categories, sections, names, repairs, and the old v1 search and details | rechecked | the store's max-age, else snap-revalidate-ttl |
| refresh and install requests | relayed; each result recorded for answering offline | none |
| download redirects without a hash, such as resources | followed and streamed, not kept | none |
| logins, device sessions and serials, cohorts, metadata | relayed | none |
| any other store path | relayed | none |
The store's max-age wins whether it is shorter or longer than the window; a max-age of zero counts as none.
Go modules
| Entry | Policy | Window |
|---|---|---|
.mod and .zip of a version | kept forever, after the checksum database check | none |
list, @latest and .info | fetched again in full | go-revalidate-ttl |
| a 404 from the upstream | not kept | none |
Docker Hub
| Entry | Policy | Window |
|---|---|---|
| layers and configs, by digest | kept forever, digest checked | none |
| manifests and indexes, by digest | kept forever, digest checked | none |
| the digest a tag resolves to, per set of accepted media types | rechecked with a HEAD | docker-revalidate-ttl |
a HEAD of a layer not yet cached | answered from the registry's HEAD, not kept | none |
| referrers | relayed | none |
LXD images
| Entry | Policy | Window |
|---|---|---|
streams/v1/index.json and every other .json catalog | fetched again in full | lxd-revalidate-ttl |
| an image file whose path a catalog listed | kept forever, hash checked | none |
| a path no catalog listed | answered 404, never fetched | none |
LXD's image API, on the HTTPS port, reads the same catalogs and files, so it follows the same policy.
Files by checksum
Every file is kept forever under its checksum and never rechecked; there is nothing to revalidate.
for administrators for CI operators
The windows
| Setting | Default | Window for |
|---|---|---|
revalidate-ttl | 5m | every side whose own setting is unset |
apt-revalidate-ttl | revalidate-ttl | apt indexes |
snap-revalidate-ttl | revalidate-ttl | snap details and assertions without a max-age |
go-revalidate-ttl | revalidate-ttl | Go version lists, @latest, .info |
docker-revalidate-ttl | revalidate-ttl | Docker tags |
lxd-revalidate-ttl | revalidate-ttl | LXD catalogs |
min-forced-revalidate-interval | 30s | apt only: how long after a check a client's request for a fresh copy is still served from the cache |
Each is a -flag, a PKGPROXY_ variable and a snap option of the same name; see Settings. A window must be longer than zero: pkgproxy refuses 0 at start and in -check-config. In offline mode every window is endless; see Offline mode.
A client asking for a fresh copy
| Side | Cache-Control: no-cache, max-age=N, Pragma: no-cache |
|---|---|
| apt | a conditional request upstream before serving, unless the index was checked within min-forced-revalidate-interval; requests forcing one index share one check |
| snap, Go, Docker, LXD, files | ignored: the window decides |
A Pragma: no-cache counts only when there is no Cache-Control. A check confirms or replaces an index; it never deletes one. To throw a cached apt file away, use pkgproxyctl cache invalidate.
When a recheck fails
| Side | Upstream unreachable, stalled or answering 5xx | Upstream says the entry is gone |
|---|---|---|
| apt | cached copy served with a warning; waits at most 10 s, host skipped for 1 min | 404 or 410 relayed as 404 |
| Go, LXD | cached copy served with a warning; waits at most 10 s, host skipped for 1 min | cached copy served with a warning, as for any answer other than 200 |
| snap | cached copy served with a warning, also on a 429 from the store | upstream's answer relayed |
| Docker | stale tag served with a warning; a 401 or 403 also keeps a tag that resolved before | 404, and the client falls back to Docker Hub |
An outcome of stale in the access log marks each such answer.
In the documentation
- Settings: the revalidate TTLs
- Removing a cached file early
Roads from here
CORE · the path every side shares
- Offline mode — freshness frozen while offline
- Stalled upstreams — when an upstream counts as stalled
- Fetch once — how a miss is fetched once
APT
- pool forever and dists rechecked — apt's pool and dists in detail
DOCKER
- Docker Hub registry mirror — tags resolved with HEAD
OPERATIONS
- Settings — changing the TTLs