HTTPS apt sources
apt for users for administrators
apt sends CONNECT for every https source to its http proxy; pkgproxy relays it as a blind tunnel to port 443, never to this machine itself, and caches nothing from it.
On the map: apt
01 What this topic covers
How apt's https sources pass through pkgproxy as an uncached tunnel, and how to get them cached instead.
When apt uses pkgproxy, its https sources pass through it in a blind tunnel, and nothing in them is cached. It helps to have seen A forward proxy first. Here you see why https sources reach the proxy at all, and the tunnel to port 443. Then why pkgproxy does not simply refuse it, and how to get such a source cached.
02 Why https sources reach the proxy
apt sends CONNECT host:443 to whatever
Acquire::http::Proxy names for any https://
source; no Acquire::https::Proxy is needed to trigger it.
A machine's sources can mix plain http archives with an https one, often a third party repository. With only the Acquire::http::Proxy line set, apt still sends its https requests to that proxy, as a CONNECT to the host on port 443. The only way to keep this traffic away from pkgproxy would be a separate https proxy setting on every machine. So pkgproxy has to answer CONNECT sensibly.
03 A blind tunnel to port 443
pkgproxy dials first, then takes over the client's connection and copies
bytes both ways. No TLS termination, no caching, no verification. A dial
that fails gets 502; one that reaches this machine's loopback
gets 403. Other ports get 403: pkgproxy only
tunnels CONNECT to port 443.
For a CONNECT to port 443, pkgproxy first dials the host. If that fails, apt gets a 502. If the name leads to this machine's own loopback, apt gets a 403: the check is made on the address actually dialled, so no name gets around it. Otherwise pkgproxy answers 200 Connection Established and copies bytes in both directions until either side closes. The TLS session runs between apt and the origin: pkgproxy does not terminate it, cannot read it, and caches nothing. A CONNECT to any other port gets a 403.
04 Why not refuse it for administrators
Refusing CONNECT (with 501) would make
apt update fail on any machine with one https-only source. The
port-443 tunnel keeps the non-goal: no TLS termination, no HTTPS caching.
Neither the tunnel nor plain http reaches this machine's loopback or
pkgproxy's own ports.
Caching HTTPS is outside pkgproxy's scope, but refusing CONNECT is not an option. A refusal makes apt update fail outright on any machine with a single https-only source, which is common for third party repositories. So pkgproxy opens the tunnel and keeps its scope: it does not terminate TLS or cache HTTPS content. The tunnel goes only to port 443, and like plain http it never reaches this machine's loopback or pkgproxy's own ports, while a mirror elsewhere on the network still works.
05 Getting such a source cached
A tunnelled request is logged with outcome=tunnel. Switch the
source to http:// where the archive offers it; apt checks
signatures either way. https-only archives keep working, uncached.
Tunnelled downloads are not cached, and the log marks each one with the outcome tunnel. If the archive is also served over plain http, change the source to http, and pkgproxy caches it like any other. That does not weaken apt's checks: apt verifies the signature on InRelease and the hashes below it either way. An archive that offers only https keeps working through the tunnel, uncached.
06 Where to go next for administrators in place of the general version
Next: The access log, for finding tunnels; pool and dists, for what an http source gets; Files by checksum, for https that pkgproxy fetches itself.
Your https sources keep working through the tunnel, uncached, and a source you switch to http is cached like the rest. To find the tunnels in the log and see which sources to switch, watch The access log. To see what such a source gets from the cache, watch pool and dists. To see where pkgproxy fetches https itself, watch Files by checksum.
07 Where to go next general version
Next: A forward proxy, for plain http sources; pool and dists, for what they get from the cache; Files by checksum, for https that pkgproxy fetches itself.
Your https sources keep working through the tunnel, uncached, and a source you switch to http is cached like the rest. To see how plain http sources reach pkgproxy, watch A forward proxy. To see what such a source gets from the cache, watch pool and dists. To see where pkgproxy fetches https itself, watch Files by checksum.
Read more
In the documentation
Roads from here
START HERE
- Why pkgproxy — what pkgproxy is for
APT
- apt through a forward proxy — how plain http sources reach pkgproxy
- pool forever and dists rechecked — what an http source gets instead
FILES
- Files by checksum — where pkgproxy fetches https itself
OPERATIONS
- The access log — the tunnel outcome in the log