pkgproxy design: setup page

The setup page the daemon serves for remote clients, and the desktop launcher that opens it. Part of the pkgproxy design, whose index says which file holds each § number.

• 20. Setup page for remote clients
  • 20.1 Routes
  • 20.2 Which address the output names
  • 20.3 Language
  • 20.4 Trust
  • 20.5 Desktop launcher
  • 20.6 Offline notice
  • 20.7 Cache status

20. Setup page for remote clients

pkgproxyctl setup <tool> (§14) only helps on a machine that has pkgproxy installed. For someone who wants to use an instance on their LAN, or a public one, the daemon serves a page at http://host:3142/ that says how. Without it that address would be a 404: the gateway sends the snap relay only /v2, /api/v1 and /v1 (§11), after the Go, LXD and Docker sides, and answers any other path itself.

20.1 Routes

The page wraps the plain listener’s handler (cmd/pkgproxy/setuppage.go). It takes only GET and HEAD of origin-form requests for exactly /, anything under /setup/ and, from a local visitor, exactly /pkgproxy/cache (§20.7); forward-proxy requests (CONNECT, absolute URIs) and every other path go through to internal/gateway unchanged. No client protocol pkgproxy serves uses these paths: snapd’s API is under /v2/ and /api/, Docker’s under /v2/, Go’s and LXD’s under their base paths.

• GET / is the landing page: HTML when Accept ranks text/html above text/plain (a browser), a plain-text rendering through guide (§22) otherwise (curl sends */*).
• GET /setup/<tool> is a shell script, and ?undo=1 its undo counterpart, for curl -fsSL http://host:3142/setup/apt | sudo sh. It is #!/bin/sh followed by exactly the lines pkgproxyctl setup <tool> prints when piped: both call the same per-tool function, so they cannot drift apart. It is served as text/plain, not text/x-shellscript, so that a browser shows it to be read rather than downloading it.

The page offers apt, snap, go, docker and lxd-remote, each only when its side is enabled (docker unless -docker-upstream off, lxd-remote only with remotes configured and the HTTPS listener on). setup lxd and setup snapcraft are left out: their scripts call pkgproxy’s local admin socket (§16.9) and ask the LXD of the machine they run on for its bridge, so they cannot work from another machine (gitlab.com/zygoon/pkgproxy#51).

It is on by default (-setup-page=false turns it off): it reveals nothing that an open forward proxy does not already.

The HTML page is self-contained, with no requests for anything but itself: the visitor may not yet reach anything but this proxy. Its styles and copy-button script are inline, and the application icon (data/pkgproxy.svg, the snap’s icon, embedded by package data) is inlined as a data: URI for both the favicon and the header.

20.2 Which address the output names

The commands must name an address the visitor can reach. -addr usually cannot be that: its host is a wildcard, and proxyHost’s substitute, the machine’s hostname, may not resolve elsewhere (mDNS often does not). The request’s Host header is the address the visitor just reached pkgproxy at, so it is used, in place of -addr, as input to the same URL helpers the CLI uses — but only when isPlainHostPort accepts it: a DNS name of letters, digits, dots and hyphens, an IPv4 address or a bracketed IPv6 address without a zone, and an optional decimal port. It ends up inside shell commands, so anything else (including no Host, as over a UNIX socket) falls back to -addr. -public-url http://host[:port] overrides both, for an instance reached through a name or port that differs from what clients send, such as behind NAT. The LXD remote URL keeps the port of -lxd-tls-addr on whichever host was chosen.

20.3 Language

The prose follows Accept-Language, matched by l10n.AcceptLanguage against the languages with a catalog. The locale is put only on a context used to render the response, never on the request’s: a request context carrying the client’s locale would put that request’s log lines in the client’s language (gitlab.com/zygoon/pkgproxy#28).

20.4 Trust

The scripts travel over plain HTTP, so piping one to sh trusts the network between the visitor and pkgproxy, exactly as using the proxy at all does. The page says so, links each script so that it can be read first, and lists the plain commands under each one-liner, for setup and for undo alike, for anyone who would rather paste them, or has no curl, as minimal Debian and Ubuntu images do not.

20.5 Desktop launcher

The snap has an applications menu entry that opens the setup page. Its Exec is pkgproxy.open, an app of its own running pkgproxyctl open with only the desktop plug: the pkgproxyctl app, often run with sudo, gets no session-bus access. snapd’s desktop file sanitizer accepts an Exec that names one of the snap’s apps and an Icon path under ${SNAP}; the translated Comment, GenericName and Keywords live in the file itself, since a desktop file has no catalog.

pkgproxyctl open asks the daemon for /pkgproxy/v1/info over the stats socket (§13.5), retrying briefly while the socket is not there yet (right after install); a daemon that omits setup_page, as an older one during a refresh does, counts as serving the page. It turns addr into a URL for this machine (a wildcard host becomes loopback) and ignores -public-url, which is how other machines reach pkgproxy and may not work from this one (NAT without hairpinning). It then runs xdg-open, which in a strict snap is snapctl user-open, handing the URL to the desktop portal or snapd’s userd; the desktop interface allows both, and no network plug is needed, as nothing is probed over TCP.

Failures go to stderr, worded “cannot …” with the fix, and exit 1. A menu entry has no terminal to show them, so the help and the README say to run pkgproxy.open in a terminal. A desktop notification is left out, as it would need a D-Bus client.

20.6 Offline notice

While pkgproxy refuses upstream requests (the mode “offline”, or an open or half-open breaker in “store-overloaded”, §21) the page shows a notice (setupPage.Notice, its own template block), rendered at each request without JavaScript: what fails and that the visitor may try later or undo the setup. “store-overloaded” with every breaker closed shows none. It names the hosts left alone and when each is tried again, nothing else of the requests that opened them: they tell a visitor whether their tools are affected, and a request through that host would say as much with a 503. It names no command, since a visitor cannot change the mode.

20.7 Cache status

On the machine that runs pkgproxy, the cache page at /pkgproxy/cache shows how the cache is doing: its size on disk, the free space of its file system (statfs(2), Linux only), the downloads saved (stats.Recorder.BytesSaved, content only: §13.5), the metadata served besides in one sentence when there is any (stats.Recorder.MetadataServed), a table per kind with the downloads saved by each, and a treemap of what fills the cache. The setup page is for setting up clients and stays light: for a local visitor it has one line under the introduction, the cache’s size and its disk’s free space, linking to the cache page (“See what is cached.”), which is how someone who opened the setup page from the launcher (§20.5) finds it. The plain-text renderings carry the same line, with the cache page’s URL at the address the visitor used (not -public-url’s, which may not work from this machine), and the cache page’s summary and table.

The cache page is under /pkgproxy/, pkgproxy’s own part of the URL space (gitlab.com/zygoon/pkgproxy#75 moves its API there), rather than at a bare /cache: no client protocol uses /pkgproxy/, but -go-base-path or -lxd-base-path could well be set to /cache.

internal/cacheusage.Scan walks -cache-dir summing allocated blocks, as du does, each object with its .meta.json, without following symlinks or crossing file systems; files outside a store’s objects/ count as other pkgproxy data. Counting on write and delete would still need a walk at start, races on overwrites and misses what bypasses store.Store. A walk is cheap only while the inodes are cached, so cacheusage.Monitor walks lazily, in the background: a view of either page starts one when the last is over a minute old and waits for it up to 1 s, otherwise it shows the previous report with its age and says it is measuring again, or “measuring” before the first. The page says the sizes are as of that walk; the downloads saved are counted as they happen. Nothing is walked while nobody looks.

Each kind is split into groups and leaves: apt by archive and source package; snaps by name and revision, from the recorder’s hash associations (§13.3); Go by module and version; LXD by image and serial, labelled with the remote and release (pkgproxy-ubuntu noble), the whole path in the tooltip; Docker blobs by the repository they were served through most, as they are keyed by digest (§19.4).

What a report keeps is folded within each kind and group, not as a share of the whole cache, so that a small kind (a hundred Go modules beside gigabytes of snaps) still shows its own largest members. Rectangles are shared out by area: a kind gets MapBudget (300) times its share of the cache, at least MinKindBudget (32); it keeps at most half that many groups, and at most MaxGroups (48); a group gets the kind’s budget times its share of the kind for its leaves and their rest, and is drawn whole below two. Within those counts the largest are kept while they take at least an eighth of their parent’s average rectangle, and the rest becomes one “N more items” rectangle. A map thus has at most 1.5 × (300 + 6 × 32) + 12 = 750 rectangles, a page of about 125 KB; a real-like cache of about 980 items draws about 240, a page of 48 KB. A long tail (hundreds of small source packages in one archive) still ends up in its group’s rest.

internal/treemap.Squarify lays out each level and the page inlines it as SVG without JavaScript: a <title> tooltip per rectangle, a <title> and <desc> summary, and the table as its visible text alternative and legend, in the Okabe–Ito palette in both themes. A kind’s label is its name and size, or its name alone when both do not fit, never cut mid-word; the tooltip and the table give the size.

What the cache holds is what the machine downloads, so only a local visitor sees either the line or the page, one who could read the stats socket anyway: over TCP, never a UNIX socket listener (VM guests and other snaps use those), from a loopback or own address, without Forwarded, X-Forwarded-* or X-Real-IP (a reverse proxy on the host makes everyone look local), for a Host that is an IP address, localhost or the host name, against DNS rebinding. Anyone else’s request for /pkgproxy/cache goes to the gateway, which answers 404 as for any path it does not know, so the page’s existence is not revealed either. Answers with the cache’s figures are Cache-Control: no-store.