One listener
core for administrators for CI operators
How one TCP port or UNIX socket serves every client, which other sockets the daemon opens, how its addresses are checked, and what answers a request before the apt, snap, Go, Docker, LXD and files sides.
On the map: core
01 What this topic covers
The port or socket every client connects to, the other sockets the daemon opens, and what answers a request before the sides.
Every client reaches pkgproxy through one listener: a TCP port, 3142 by default. It helps to have seen A request first. Here you see the UNIX socket that can replace that port, and the other sockets the daemon opens beside it. Then why every socket is world writable, how the addresses are checked before anything binds, and what answers a request before it reaches the apt, snap, Go, Docker, LXD or files side.
02 A TCP port, or a UNIX socket
-addr (default :3142) is the TCP listener.
-socket-path replaces it with a UNIX socket, for example for
QEMU's guestfwd. Setup instructions always name
-addr.
Every client reaches pkgproxy through one listener. By default it is TCP on port 3142, on every address, set with the addr flag. The -socket-path flag replaces it with a UNIX socket, for a host that wants no TCP port at all, such as a proxy bridged into a virtual machine through QEMU. The setup instructions still name the addr flag either way: they describe where clients connect, not where the daemon binds.
03 The other sockets the daemon opens
-lxd-tls-addr (:3143) is HTTPS for lxc.
stats.sock and admin.sock live in
/var/snap/pkgproxy/common in the snap, /run/pkgproxy
otherwise. The snap adds content sockets under
$SNAP_COMMON/sockets.
The daemon opens more than the main port. An HTTPS listener on 3143 serves LXD image remotes, with a self signed certificate kept in the cache directory. Two UNIX sockets serve pkgproxyctl: the stats socket, which stats, open and setup read, and the admin socket, which changes the running daemon, for cache invalidate and offline. In the snap, content sockets serve both listeners again, to share the cache with another snap, such as image-garden.
04 World-writable sockets, on purpose for administrators
bind(2) applies the umask; connect(2) needs
write permission. The umask is cleared around every bind, so each socket
is 0777. A stale socket file is removed first; this is not a
check for a second daemon.
Creating a UNIX socket applies the process umask, like creating any file. Under snapd's umask of 0022 a socket would end up 0755, and connecting to a socket needs write permission, so an unprivileged process at the other end of a bridge could not connect. pkgproxy clears the umask around the bind, for every socket it opens, leaving each one 0777. That is deliberate: anyone who can reach these sockets on this host is trusted to use them. A stale socket file left by a previous run is removed before binding.
05 Addresses are checked before anything binds for administrators
-addr and -lxd-tls-addr must be
host:port with a numeric port. pkgproxy
-check-config validates flags and PKGPROXY_*
variables and exits; the snap's configure hook runs it.
The addr and LXD TLS addr values are checked before anything binds. Each must be host and port: the host empty, an IP address or a host name, and the port a number from zero to 65535. A service name such as http is refused, because the setup scripts must name the port. A host name is not resolved at this point. The check config flag runs these checks and exits, and the snap's configure hook runs it, so a bad snap set fails at once instead of leaving the service in a restart loop.
06 What answers before the sides
Main port: access log, setup page (/, /setup/,
/pkgproxy/cache), LXD certificate (/pkgproxy-lxd-
cert.pem), offline mode, then the sides. HTTPS port: access log,
LXD image API, offline mode, the same sides.
On the main port, a request passes a few layers before it is handed to the apt, snap, Go, Docker, LXD or files side. The access log wraps everything, so every request gets one line. The setup page answers slash, the scripts under /setup, and the cache page for a local visitor. Next comes the download of the LXD certificate, then offline mode, which answers what the cache cannot with a 503. The HTTPS port has its own chain: the access log, LXD's image API, which LXD 6.10 and later download through, then offline mode and the same sides.
07 Where to go next general version
Next: Settings, for the address and socket path; LXD images, for the HTTPS port; The cache page, for what the main port shows a local visitor.
You now know where every client comes in, and what answers a request before the sides do. To change the address, the socket path and the rest, watch Settings. To see what the HTTPS port is for, watch LXD images. To see the page the main port shows a visitor on the same machine, watch The cache page.
Read more
The main port, 3142
Plain HTTP on -addr, or on the UNIX socket -socket-path names instead. A request is offered to each row in turn; the first that claims it answers. The name in the last column is what the access log records as the handler.
| Order | Request | Answered by | Local or trusted only | Log name |
|---|---|---|---|---|
| 1 | GET or HEAD / | the setup page | no | setup-page |
| 1 | GET or HEAD /setup/TOOL, ?undo=1 for the undo script | a setup script for sh | no | setup-page |
| 1 | GET or HEAD /pkgproxy/cache | the cache page | yes | setup-page |
| 1 | GET or HEAD /pkgproxy/cache/activity | the cache page's figures, as JSON | yes | setup-page |
| 1 | GET or HEAD /pkgproxy/cache/events | the cache page's live stream | yes | setup-page |
| 2 | GET or HEAD /pkgproxy-lxd-cert.pem | the HTTPS port's certificate, in PEM | no | lxd-cert |
| 3 | CONNECT host:port, or an absolute URL such as GET http://archive.ubuntu.com/… | the apt forward proxy | no | apt |
| 4 | /go and below | the Go module proxy | no | gomod |
| 5 | /lxd/NAME/… | an LXD image server, by simplestreams | no | lxd |
| 6 | /files/v1/fetch?… | files by checksum | no | files |
| 7 | /v2/, /v2/REPO/manifests/REF, /v2/REPO/blobs/DIGEST, /v2/REPO/referrers/DIGEST | the Docker Hub mirror | no | docker |
| 7 | /v2/REPO/tags/list, /v2/_catalog | the Docker Hub mirror, which answers 404 | no | docker |
| 8 | any other path under /v2, /api/v1 or /v1 | the Snap Store relay | no | snap |
| 9 | anything else | 404, sent nowhere | no | (none) |
- Rows 1 and 2 take only
GETandHEAD; other methods fall through. Row 1 goes away withsetup-pageset tofalse, row 2 when the HTTPS port is off. - A cache path asked by anyone else falls through and gets the 404 of row 9. Allowed means: over TCP, not a UNIX socket; from a loopback address, one of the machine's own or one in
trusted-networks(-trusted-networks, empty by default); without a reverse proxy's header such asViaorX-Forwarded-For; and with aHostthat is an IP address,localhost, the machine's name or the host ofpublic-url. - The Go and LXD paths move with
-go-base-pathand-lxd-base-path. A side that is turned off claims nothing: its paths reach row 9, and Docker-shaped paths get a 404 rather than going to the Snap Store. - From row 3 on, offline mode answers what the cache cannot serve; see Offline mode.
The HTTPS port, 3143
HTTPS on -lxd-tls-addr, with the self-signed certificate from the cache directory, or on the UNIX socket -lxd-tls-socket-path names. It exists for lxc, which talks to image servers only over HTTPS. It is off when -lxd-tls-addr is off or no LXD remote is configured.
| Order | Request | Answered by | Log name |
|---|---|---|---|
| 1 | GET / | LXD's API root, as LXD asks when it adds a remote | lxd-api |
| 1 | GET /1.0 | the server description | lxd-api |
| 1 | GET /1.0/images?project=NAME | the images of the remote NAME | lxd-api |
| 1 | GET /1.0/images/aliases/ALIAS?project=NAME | an alias of that remote | lxd-api |
| 1 | GET /1.0/images/FINGERPRINT?project=NAME | one image | lxd-api |
| 1 | GET /1.0/images/FINGERPRINT/export?project=NAME | the image's files, as LXD 6.10 and later download them | lxd-api |
| 2 | anything else | the same sides as the main port, from row 3 on | as on the main port |
Under /1.0, other paths get LXD's 404 and other methods a 405. The setup page, the cache page and the certificate download are on the main port only. The lxc remote add commands that pkgproxyctl setup lxd-remote prints use https://HOST:3143/lxd/NAME with simplestreams.
The UNIX sockets
| Socket | Snap | Without the snap | Serves | Used by |
|---|---|---|---|---|
| stats | /var/snap/pkgproxy/common/stats.sock | /run/pkgproxy/stats.sock | GET /pkgproxy/v1/stats, GET /pkgproxy/v1/info | pkgproxyctl stats, open and setup |
| admin | /var/snap/pkgproxy/common/admin.sock | /run/pkgproxy/admin.sock | DELETE /pkgproxy/v1/cache with the apt URL in a JSON body, GET and PUT /pkgproxy/v1/offline | pkgproxyctl cache invalidate, pkgproxyctl offline, the snap's configure hook |
| content, main | /var/snap/pkgproxy/common/sockets/pkgproxy.sock | where -content-socket-path points | the main port's table, without the cache page's rows | another snap, through the sockets content interface |
| content, HTTPS | /var/snap/pkgproxy/common/sockets/pkgproxy-lxd-tls.sock | where -lxd-tls-content-socket-path points | the HTTPS port's table | the same |
- A content socket is in addition to the TCP port;
-socket-pathand-lxd-tls-socket-pathreplace it. - Every socket is made
0777: anyone on the machine may connect, so anyone may switch offline mode or drop an apt file. The stats socket only reads. - A stale socket file from an earlier run is removed before binding, and the parent directory is made if missing.
See Statistics for the stats API, pkgproxyctl for the commands and Settings for the flags.
In the documentation
- Settings: the ports, sockets and addresses
- After changing a port
Roads from here
START HERE
- A request, end to end — a request end to end
CORE · the path every side shares
- Fetch once — what a side does with a miss
LXD
- LXD image servers — what the HTTPS port is for
SETUP AND THE COMMAND LINE
- The setup page — the page the main port shows
OPERATIONS
- Settings — changing the address and other settings
- Statistics — what the stats socket serves
- The cache page — the page a local visitor gets