One listener

core for administrators for CI operators

How one TCP port or UNIX socket serves every client, which other sockets the daemon opens, how its addresses are checked, what answers a request before the apt, snap, Go, Docker, LXD and files proxy engines, and the health route in front of them all.

On the map: core

▶ Watch this deck — 8 scenes, ~5 min

Part of journeys

01 What this topic covers

ABOUT THIS TOPIC One listener the one port, or socket, where every client reaches pkgproxy 1 A TCP port, or a UNIX socket 2 The other sockets the daemon opens 3 World-writable sockets, on purpose for administrators 4 Addresses are checked before binding for administrators 5 What answers before the proxy engines 6 A health route in front of it all
The topic title, One listener, and its 6 scenes in order: A TCP port, or a UNIX socket, The other sockets the daemon opens, World-writable sockets, on purpose, Addresses are checked before binding, What answers before the proxy engines, A health route in front of it all

The port or socket every client connects to, the other sockets the daemon opens, what answers a request before the proxy engines, and the health route.

Every client reaches pkgproxy through one listener: a TCP port, 3142 by default. It helps to have seen A request first. Here you see the UNIX socket that can replace that port, and the other sockets the daemon opens beside it. Then why every socket is world writable, how the addresses are checked before anything binds, and what answers a request before it reaches the apt, snap, Go, Docker, LXD or files proxy engine. Last comes the health route, in front of everything else.

02 A TCP port, or a UNIX socket

ONE LISTENER TCP apt docker lxc :3142 pkgproxy -addr :3142 the default, every address UNIX socket apt in a VM guest QEMU guestfwd, unix: form UNIX socket at <path> pkgproxy -socket-path <path> replaces the TCP listener setup scripts name http://<host>:3142 from -addr, in both modes
Left: clients connect to pkgproxy over TCP on port 3142. Right: a VM guest reaches pkgproxy through QEMU's guestfwd and a UNIX socket. Below: the setup scripts name the -addr address in both cases

-addr (default :3142) is the TCP listener. -socket-path replaces it with a UNIX socket, for example for QEMU's guestfwd. Setup instructions always name -addr.

Every client reaches pkgproxy through one listener. By default it is TCP on port 3142, on every address, set with the addr flag. The -socket-path flag replaces it with a UNIX socket, for a host that wants no TCP port at all, such as a proxy bridged into a virtual machine through QEMU. The setup instructions still name the addr flag either way: they describe where clients connect, not where the daemon binds.

03 The other sockets the daemon opens

EVERY LISTENER pkgproxy :3142 · TCP every client and the setup page :3143 · HTTPS LXD image remotes <cache-dir>/lxd/tls stats.sock read: stats · open · setup admin.sock change: cache invalidate · offline pkgproxyctl snap only: the same listeners again, for another snap $SNAP_COMMON/sockets/pkgproxy.sock $SNAP_COMMON/sockets/pkgproxy-lxd-tls.sock
pkgproxy in the middle with its listeners: TCP 3142 for every client, HTTPS 3143 for lxc, the stats and admin UNIX sockets used by pkgproxyctl, and two content sockets shared with another snap

-lxd-tls-addr (:3143) is HTTPS for lxc. stats.sock and admin.sock live in /var/snap/pkgproxy/common in the snap, /run/pkgproxy otherwise. The snap adds content sockets under $SNAP_COMMON/sockets.

The daemon opens more than the main port. An HTTPS listener on 3143 serves LXD image remotes, with a self signed certificate kept in the cache directory. Two UNIX sockets serve pkgproxyctl: the stats socket, which stats, open and setup read, and the admin socket, which changes the running daemon, for cache invalidate and offline. In the snap, content sockets serve both listeners again, to share the cache with another snap, such as image-garden.

04 World-writable sockets, on purpose for administrators

UMASK bind under snapd's umask umask 0022 srwxr-xr-x 0755 unprivileged bridge process connect needs write: refused pkgproxy clears the umask around every bind umask 0 srwxrwxrwx 0777 unprivileged bridge process connects a trust boundary of one host a stale socket from the last run is removed before the bind
Two rows. With snapd's umask 0022 the socket is created 0755 and an unprivileged process cannot connect. With the umask cleared around the bind the socket is 0777 and it can

bind(2) applies the umask; connect(2) needs write permission. The umask is cleared around every bind, so each socket is 0777. A stale socket file is removed first; this is not a check for a second daemon.

Creating a UNIX socket applies the process umask, like creating any file. Under snapd's umask of 0022 a socket would end up 0755, and connecting to a socket needs write permission, so an unprivileged process at the other end of a bridge could not connect. pkgproxy clears the umask around the bind, for every socket it opens, leaving each one 0777. That is deliberate: anyone who can reach these sockets on this host is trusted to use them. A stale socket file left by a previous run is removed before binding.

05 Addresses are checked before anything binds for administrators

CHECKED FIRST accepted :3142 127.0.0.1:3142 [::1]:3142 proxy.example:3142 [::]:3143 refused: a service name is not a port number $ pkgproxy -check-config -addr :http pkgproxy error: cannot use -addr ":http": the port must be a number from 0 to 65535 a host name is not resolved here; one that does not resolve fails at start snap set pkgproxy … configure hook pkgproxy -check-config restart if needed invalid: snap set fails at once no restart loop from a bad value
Accepted listen addresses on the left, a refused service-name port in a terminal on the right, and the snap configure hook running the same check before the daemon restarts

-addr and -lxd-tls-addr must be host:port with a numeric port. pkgproxy -check-config validates flags and PKGPROXY_* variables and exits; the snap's configure hook runs it.

The addr and LXD TLS addr values are checked before anything binds. Each must be host and port: the host empty, an IP address or a host name, and the port a number from zero to 65535. A service name such as http is refused, because the setup scripts must name the port. A host name is not resolved at this point. The check config flag runs these checks and exits, and the snap's configure hook runs it, so a bad snap set fails at once instead of leaving the service in a restart loop.

06 What answers before the proxy engines

LAYERS :3142 access log one line each setup page / /setup/… /pkgproxy/status /pkgproxy/cache… /pkgproxy/upstreams… the last two: local and trusted LXD cert /pkgproxy-lxd-cert.pem offline 503 for what is not cached loop check 508 for a loop dispatch apt · snap go · docker lxd · files :3143 access log LXD image API LXD 6.10 and later offline both ports end in the same dispatch to the proxy engines each layer answers its own paths and passes every other request on
Two chains of layers. Main port: access log, setup page, LXD certificate download, offline mode, a loop check, then dispatch to the proxy engines. HTTPS port: access log, LXD image API, offline mode, the same dispatch

Main port: access log, setup page (/, /setup/, /pkgproxy/status, /pkgproxy/cache, /pkgproxy/upstreams), LXD certificate, offline mode, the loop check (508), then the proxy engines. HTTPS port: access log, LXD image API, offline mode, the same proxy engines.

On the main port, a request passes a few layers before it is handed to the apt, snap, Go, Docker, LXD or files proxy engine. The access log wraps everything. The setup page answers slash, the scripts under /setup, the status in every page's bar, and the cache and upstream pages for a local visitor. Next comes the download of the LXD certificate, then offline mode, which answers what the cache cannot with a 503, then a check that turns away a request looping between package proxies. The HTTPS port has its own chain: the access log, LXD's image API, then offline mode and the same proxy engines.

07 A health route in front of it all

HEALTH /pkgproxy/v1/health access log … the proxy engines a probe a service manager a load balancer a Kubernetes probe GET or HEAD on :3142, or the socket in its place, and on the content socket $ curl http://localhost:3142/pkgproxy/v1/health {"status":"ok","mode":"store-overloaded"} and "automatic": "no-network" while offline by itself offline is still 200: the cache serves what it holds served with the setup page turned off names no upstreams: anyone who reaches the port may ask never written to the access log
A probe asks /pkgproxy/v1/health, which is answered in front of the access log and the other layers; a curl of it gets 200 with the status and the offline mode; offline is still 200; the route is served with the setup page off, names no upstreams and is never logged

GET or HEAD /pkgproxy/v1/health on -addr (or -socket-path) and the content socket: 200 and {"status":"ok","mode":"store-overloaded"}, with automatic while offline by itself. Served with -setup-page=false; not in the access log.

In front of all those layers, even the access log, sits one route: /pkgproxy/v1/health. A service manager, a load balancer or a Kubernetes probe asks it with GET or HEAD, on the main port or the socket that replaces it, and on the content socket. While pkgproxy serves, it answers 200, with the offline mode, and why, if pkgproxy went offline by itself. Offline is still healthy, as the cache serves what it holds. The route stays with the setup page turned off, names no upstreams, and never reaches the access log.

08 Where to go next general version

WHERE NEXT you know where every client comes in, and what answers a request before the proxy engines do you are here One listener Settings the address, the socket path and the rest LXD images what the HTTPS port is for The cache page what the main port shows a local visitor
You are here, One listener; next: Settings, LXD images, The cache page

Next: Settings, for the address and socket path; LXD images, for the HTTPS port; The cache page, for what the main port shows a local visitor.

You now know where every client comes in, and what answers a request before the proxy engines do. To change the address, the socket path and the rest, watch Settings. To see what the HTTPS port is for, watch LXD images. To see the page the main port shows a visitor on the same machine, watch The cache page.

Read more

The main port, 3142

Plain HTTP on -addr, or on the UNIX socket -socket-path names instead. A request is offered to each row in turn; the first that claims it answers. The name in the last column is what the access log records as the handler.

OrderRequestAnswered byLocal or trusted onlyLog name
0GET or HEAD /pkgproxy/v1/healththe health route: 200 while pkgproxy servesnonot logged
1GET or HEAD /the setup pagenosetup-page
1GET or HEAD /setup/TOOL, ?undo=1 for the undo scripta setup script for shnosetup-page
1GET or HEAD /pkgproxy/cachethe cache pageyessetup-page
1GET or HEAD /pkgproxy/cache/activitythe cache page's figures, as JSONyessetup-page
1GET or HEAD /pkgproxy/cache/eventsthe cache page's live streamyessetup-page
1GET or HEAD /pkgproxy/cache/upstreamsthe status dots of the cache page's map, as JSONyessetup-page
1GET or HEAD /pkgproxy/upstreams, /pkgproxy/upstreams/HOSTthe upstream status pagesyessetup-page
1GET or HEAD /pkgproxy/statusthe status element of every page's bar, never cachednosetup-page
2GET or HEAD /pkgproxy-lxd-cert.pemthe HTTPS port's certificate, in PEMnolxd-cert
3CONNECT host:port, or an absolute URL such as GET http://archive.ubuntu.com/…the apt forward proxynoapt
3/pkgproxy/v1/proxy-engineswhich proxy engines this pkgproxy runs, and where, for a pkgproxy using it as its upstreamno(none)
4/go and belowthe Go module proxynogomod
5/lxd/NAME/…an LXD image server, by simplestreamsnolxd
6/files/v1/fetch?…, /files/v1/volatile?…files by checksum, and files that change under their URLnofiles
7/v2/, /v2/REPO/manifests/REF, /v2/REPO/blobs/DIGEST, /v2/REPO/referrers/DIGESTthe Docker Hub mirrornodocker
7/v2/REPO/tags/list, /v2/_catalogthe Docker Hub mirror, which answers 404nodocker
8any other path under /v2, /api/v1 or /v1the Snap Store relaynosnap
9anything else404, sent nowhereno(none)
  • Row 0 sits in front of the access log, so a probe every few seconds leaves no lines; it is served whatever setup-page says. A forward-proxy request for the same path goes upstream as any other.
  • Rows 0, 1 and 2 take only GET and HEAD; other methods fall through. Row 1 goes away with setup-page set to false, row 2 when the HTTPS port is off.
  • A cache path asked by anyone else falls through and gets the 404 of row 9. Allowed means: over TCP, not a UNIX socket; from a loopback address, one of the machine's own or one in trusted-networks (-trusted-networks, empty by default); without a reverse proxy's header such as Via or X-Forwarded-For; and with a Host that is an IP address, localhost, the machine's name or the host of public-url.
  • The Go and LXD paths move with -go-base-path and -lxd-base-path. A proxy engine that is turned off claims nothing: its paths reach row 9, and Docker-shaped paths get a 404 rather than going to the Snap Store.
  • From row 3 on, offline mode answers what the cache cannot serve; see Offline mode.
  • From row 3 on, a request that carries this pkgproxy's own instance in X-Pkgproxy-Via, or more than 8, is a loop between pkgproxies, answered 508 Loop Detected; see Two tiers. Every answer of a proxy engine carries X-Pkgproxy-Engine with its name: apt, snap, go, docker, lxd, files or files-volatile.

The health route

GET /pkgproxy/v1/health is for service managers, load balancers, Pebble checks and Kubernetes probes. It is on the main port (or the socket -socket-path names in its place) and the main content socket, not on the HTTPS port.

$ curl http://localhost:3142/pkgproxy/v1/health {"status":"ok","mode":"store-overloaded"}

FieldMeaning
statusalways ok: an answer at all means pkgproxy serves; otherwise the connection is refused
modethe offline mode in effect; offline is still 200, as the cache serves what it holds
automaticonly while pkgproxy is offline by itself: no-network or airplane-mode

It names no upstreams, as anyone who reaches the port may ask it; the stats socket's /pkgproxy/v1/info lists them.

The HTTPS port, 3143

HTTPS on -lxd-tls-addr, with the self-signed certificate from the cache directory, or on the UNIX socket -lxd-tls-socket-path names. It exists for lxc, which talks to image servers only over HTTPS. It is off when -lxd-tls-addr is off or no LXD remote is configured.

OrderRequestAnswered byLog name
1GET /LXD's API root, as LXD asks when it adds a remotelxd-api
1GET /1.0the server descriptionlxd-api
1GET /1.0/images?project=NAMEthe images of the remote NAMElxd-api
1GET /1.0/images/aliases/ALIAS?project=NAMEan alias of that remotelxd-api
1GET /1.0/images/FINGERPRINT?project=NAMEone imagelxd-api
1GET /1.0/images/FINGERPRINT/export?project=NAMEthe image's files, as LXD 6.10 and later download themlxd-api
2anything elsethe same proxy engines as the main port, from row 3 onas on the main port

Under /1.0, other paths get LXD's 404 and other methods a 405. The setup page, the cache page and the certificate download are on the main port only. The lxc remote add commands that pkgproxyctl setup lxd-remote prints use https://HOST:3143/lxd/NAME with simplestreams.

The UNIX sockets

SocketSnapWithout the snapServesUsed by
stats/var/snap/pkgproxy/common/stats.sock/run/pkgproxy/stats.sockGET /pkgproxy/v1/stats, GET /pkgproxy/v1/info, GET /pkgproxy/v1/upstreams and its /history and /eventspkgproxyctl stats, upstreams, open and setup
admin/var/snap/pkgproxy/common/admin.sock/run/pkgproxy/admin.sockDELETE /pkgproxy/v1/cache with the apt URL in a JSON body, GET and PUT /pkgproxy/v1/offlinepkgproxyctl cache invalidate, pkgproxyctl offline, the snap's configure hook
content, main/var/snap/pkgproxy/common/sockets/pkgproxy.sockwhere -content-socket-path pointsthe main port's table, without the cache page's rows, the health route includedanother snap, through the sockets content interface
content, HTTPS/var/snap/pkgproxy/common/sockets/pkgproxy-lxd-tls.sockwhere -lxd-tls-content-socket-path pointsthe HTTPS port's tablethe same
  • A content socket is in addition to the TCP port; -socket-path and -lxd-tls-socket-path replace it.
  • Every socket is made 0777: anyone on the machine may connect, so anyone may switch offline mode or drop an apt file. The stats socket only reads.
  • A stale socket file from an earlier run is removed before binding, and the parent directory is made if missing.

See Statistics for the stats API, pkgproxyctl for the commands and Settings for the flags.

In the documentation

Roads from here

START HERE

CORE · the path every proxy engine shares

  • Fetch once — what a proxy engine does with a miss

LXD

SETUP AND THE COMMAND LINE

OPERATIONS