The guestfwd unix socket patch
TCP in, unix socket out for contributors
Stock QEMU's guestfwd can start a command or open one shared character device, but not a fresh unix socket connection per guest connection. image-garden's qemu-patches/guestfwd-unix-socket.patch adds that unix: form with libslirp's slirp_add_unix; the snap and the container build their QEMU with it, a distribution QEMU lacks it, and a URL works without it.
On the map: pkgproxy
▶ Watch this deck — 7 scenes, ~5 min
Worth seeing first
- How a guest reaches pkgproxy — the two ways a guest reaches pkgproxy, and where guestfwd fits
01 TCP in, unix socket out
The guest speaks TCP to 10.0.2.100:3142; the snap's pkgproxy listens on a unix socket. qemu-patches/guestfwd-unix-socket.patch lets QEMU join the two.
In the snap, pkgproxy doesn't listen on a TCP port. It listens on a unix socket, a file in the snap's data directory. The guest, though, only speaks TCP: it connects to 10.0.2.100, port 3142, and expects someone to answer. Something has to join the two, once for every connection the guest opens. QEMU's user network almost can, and that gap is where image-garden steps in. It carries one small patch for QEMU, so that its guestfwd option can send each guest connection straight to a unix socket. This deck is about that patch: what it adds, who builds with it, and what you lose without it.
Details and sources
Details
- The snap's pkgproxy service runs
pkgproxy -addr pkgproxy:3142 -socket-path $SNAP_COMMON/pkgproxy.sock …(snapcraft.yaml:241);-addronly names it in pkgproxy's printed banner, "binding is over the UNIX socket" (proxy/README.md:50-55). - "QEMU's
guestfwdforwards guest TCP connections to 10.0.2.100 on port 3142 straight to that UNIX socket … one fresh UNIX connection per guest TCP connection, handled natively inside QEMU" (proxy/README.md:16-23). - "The
unix:form ofguestfwdisn't a stock QEMU feature" (proxy/README.md:25-33). The patch is qemu-patches/guestfwd-unix-socket.patch.
Read more
- How a guest reaches pkgproxy, with the socket path drawn out.
02 What stock QEMU offers
cmd: starts a process per connection (the old nc -U bridge); a chardev shares one host connection between all guest connections. Neither gives one socket connection per guest connection.
Stock QEMU's guestfwd has two kinds of target. The first is a command: QEMU starts a process for every guest connection and passes the traffic through it. image-garden used to do exactly that, starting nc for each connection to reach the socket. It works, but it's one more program to ship, and a new process per connection. The second is a character device, and a bare unix path ends up there. But a character device is a single connection, opened once and shared by every guest connection. For HTTP, where requests run side by side, that's wrong. What's missing is in between: a fresh socket connection for each guest connection.
Details and sources
Details
- From QEMU's command line, "only
cmd:command(spawn a process per connection) and a generic chardev form (one connection shared across *all* guest connections — wrong for HTTP) were reachable" (proxy/README.md:25-31). - "Before this patch existed, the bridge instead spawned
nc -Uper connection viaguestfwd=...-cmd:...; the patch removes that external-process dependency entirely" (proxy/README.md:34-36). - The patch's own header: a bare
unix:/pathalready falls through to the chardev path (qemu_chr_new_mux_mon), which "opens one chardev shared across the whole config-line's lifetime, multiplexing every guest TCP connection onto a single host-side connection — wrong for a protocol like HTTP where concurrent requests need independent connections" (guestfwd-unix-socket.patch:15-20). SOCKETon the slide stands for the socket's path.
Read more
03 Six lines in QEMU's network code for contributors
A unix: branch in net/slirp.c passes the path, guest address and port to libslirp's slirp_add_unix(), which connects afresh for every guest connection.
The fix is small because the hard part already exists. libslirp, the library behind QEMU's user network, has a function for exactly this: slirp_add_unix, which opens a new connection to a unix socket for each guest connection. QEMU's option parser never called it. The patch adds one branch to the file net/slirp.c, after the command form. When the target starts with unix and a colon, the rest is the socket path, and the branch hands it to slirp_add_unix with the guest address and port. If that fails, you get the same error the command form gives: conflicting or invalid host and port in the guest forwarding rule.
Details and sources
Details
- The patch's header: "Add a "unix:" form to -netdev user's guestfwd=, forwarding each new guest TCP connection to a fresh AF_UNIX connection to the given host socket path (one connection per guest connection, same semantics as the existing "cmd:" exec form). libslirp already implements exactly this via slirp_add_unix()/open_unix() … it's just never wired up from QEMU's own guestfwd= option parser" (guestfwd-unix-socket.patch:6-13).
- The hunk adds six lines to
net/slirp.cat line 1153 of the QEMU source the snap and the container build, between thecmd:branch and the chardev fallback (guestfwd-unix-socket.patch:22-36).&p[5]is the text afterunix:, the socket path;serverandportare, by libslirp'sslirp_add_unix()signature, the guest address and port fromtcp:10.0.2.100:3142(derived: QEMU'snet/slirp.cand libslirp are not in the trees this map reads). - The error text is the same as the
cmd:branch's, shown as the context line above the hunk (guestfwd-unix-socket.patch:24-27). - The patch carries an SPDX header (Apache-2.0) and an author line; it is applied with
patch -p1from the QEMU source directory.
Read more
04 Who builds QEMU with it
The snap's qemu part and the container's build-qemu stage both run patch -p1 with qemu-patches/guestfwd-unix-socket.patch. A distribution's QEMU lacks it.
Two of image-garden's installs build their own QEMU, and both apply the patch. The snap builds QEMU once, from the release tarball, and its QEMU part runs the patch right after pulling the source, so every QEMU component of the snap has it. The container's build stage downloads the same tarball, checked against the same hash, copies the patch in and applies it before configuring. Both take it from the same place, the qemu-patches directory of the image-garden tree. A QEMU from your distribution doesn't have it, and neither does one you build yourself, unless you apply the patch too.
Details and sources
Details
- The snap builds QEMU once, in the
qemupart, from the release tarball with a pinnedsource-checksum;override-pullrunscraftctl defaultand thenpatch -p1 < "${CRAFT_PROJECT_DIR}"/qemu-patches/guestfwd-unix-socket.patch(snapcraft.yaml:327-341). The patch is "applied to all bundled QEMU system-emulator parts" (proxy/README.md:32-33). - The container's
build-qemustage adds the same tarball withADD --checksum=sha256:5a8289c7…(Dockerfile:5-6), then "The same patch as the snap's QEMU",COPY qemu-patches/guestfwd-unix-socket.patch /src/andRUN patch -d /src/qemu-11.1.3 -p1 </src/guestfwd-unix-socket.patch, beforeconfigure(Dockerfile:28-36). The container's NEWS records it under 0.7.0 (NEWS.docker:18-21). - The snap and the container build the same QEMU release from the same checksum (
5a8289c7…in both files). - A distribution's QEMU is built from QEMU's own sources without this patch (derived: the patch is image-garden's and not part of QEMU).
Read more
05 What you lose without it
Outside the snap, pkgproxy is reached by URL, which needs no patch. The smoke test skips the socket modes when the QEMU binary does not mention slirp_add_unix.
So what does a missing patch cost you? Less than you might think. The socket modes belong to the snap: make looks for a socket only where SNAP_COMMON points, and outside the snap nothing sets that for you. A native install, or the container, names a pkgproxy with PKGPROXY_URL instead, and a URL works with any QEMU, because the guest simply connects out. Where it matters is a test that runs the socket modes outside the snap. Only the patch calls slirp_add_unix, so image-garden's smoke test looks for that name in the QEMU binary, and skips the socket modes when it isn't there.
Details and sources
Details
- make looks for
pkgproxy.sockin$SNAP_COMMONor$SNAP_COMMON/pkgproxy-host-snap/sockets(010-qemu.mk:20-23, :31). snapd setsSNAP_COMMONfor the snap's apps; outside the snap it is unset unless you set it, as image-garden's own tests do with a fixture directory (derived from check-pkgproxy-config.sh:103-120). - "Outside the snap, as in image-garden's container,
make PKGPROXY_URL=http://host:port ...does the same, and so doesPKGPROXY_URLin the environment" (proxy/README.md:122-127). With a URL make adds noguestfwd(010-qemu.mk:24-30). - The smoke test, for
componentandsnap: "The patch is the only caller of slirp_add_unix in QEMU",grep -q slirp_add_unix "$qemu", elseSKIP: $mode needs QEMU patched with qemu-patches/guestfwd-unix-socket.patch; … is notandexit 77(smoke-pkgproxy.sh:84-90). A QEMU linked to a shared libslirp names the function it imports, so the name is in the binary only when something calls it (derived). - Given a socket path on an unpatched QEMU,
unix:PATHfalls through to the chardev path: one host connection shared by every guest connection, per the patch's description (guestfwd-unix-socket.patch:15-20). image-garden itself sets up the socket modes only in the snap, whose QEMU has the patch. - proxy/README.md's "Early-Access" section says the same: "The snap bundles pkgproxy and sets it up; outside the snap, as in image-garden's container, guests use a pkgproxy that
PKGPROXY_URLnames" (proxy/README.md:7-12).
Read more
06 How the bridge is tested for contributors
tests/check-pkgproxy-config.sh checks the guestfwd options without booting; tests/smoke-pkgproxy.sh MODE SYSTEM.ARCH boots a guest per mode; the snap's spread suite runs the smoke test nested.
Three tests keep the bridge honest, from cheap to thorough. The config check runs make for every mode, with plain files standing in for pkgproxy's sockets, and compares the guestfwd options it gets: one rule for the bundled socket, a second for the LXD one when it's there, the pkgproxy snap's sockets in snap mode, and nothing for a URL. It boots nothing, and needs no patched QEMU. The smoke test does boot: it starts pkgproxy, brings up one guest in the chosen mode, and checks that what the guest downloads lands in the cache. And the snap's spread suite runs that smoke test in every mode, nested in a VM.
Details and sources
Details
tests/check-pkgproxy-config.sh"Check[s] what image-garden generates for each way of choosing a pkgproxy, without booting anything", with$SNAP_COMMONa fixture directory and "plain files standing in for pkgproxy's sockets" (check-pkgproxy-config.sh:5-9). Its cases: component with and without the LXD socket, component down, snap, snap disconnected, external with and without a port, an unresolvable URL,PKGPROXY_URLfrom the environment, none and nothing (check-pkgproxy-config.sh:115-170). It also checks the user-data of six families withtests/check-pkgproxy-user-data.py.tests/smoke-pkgproxy.sh MODE SYSTEM.ARCH, MODE one ofcomponent,snap,url,none, runs pkgproxy on the host and boots one VM (smoke-pkgproxy.sh:5-23); "It checks the guest's setup, and that what the guest downloads ends up in pkgproxy's cache, or, fornone, does not" (proxy/README.md:247-253).- The snap's spread suite in
tests/snap/runstests/smoke-pkgproxy.shwithSMOKE_SNAP=1for each mode, "with the guest nested in the VM"; it runs only when named (proxy/README.md:261-274). - "CI runs them before the slow tests of every system" (proxy/README.md:235-238).
Read more
- How a guest reaches pkgproxy, which draws each mode's option.
07 Where to go from here
Next: reaching pkgproxy, a distribution QEMU, the container, the snap and why pkgproxy at all.
That's the whole patch: six lines that let QEMU hand each guest connection to a unix socket, carried by the snap and the container. To see where the bridge sits among the ways a guest finds pkgproxy, take the reaching deck. If you run a distribution's QEMU, the native QEMU deck says what else differs from the bundled one. The container deck shows its patched QEMU and PKGPROXY_URL at work, and the snap overview shows the QEMU components that carry the patch.
Details and sources
Read more
Read more
Basis of the statements on this page
Read in qemu-patches/guestfwd-unix-socket.patch, snap/snapcraft.yaml, Dockerfile, mk/010-qemu.mk, proxy/README.md and tests/ of image-garden at the v0.8.0 tag ("code"). QEMU's net/slirp.c and libslirp are not in the trees this map reads: what the slides say about them comes from the patch's description and proxy/README.md, and the rest is labelled "derived".
guestfwd targets
Target after tcp:ADDR:PORT- | In stock QEMU | With the patch | Host side |
|---|---|---|---|
cmd:COMMAND | yes | yes | a new process per guest connection |
unix:PATH | read as a chardev | the patch's branch | a new socket connection per guest connection |
| a chardev | yes | yes | one connection shared by every guest connection |
The snap's .run scripts use guestfwd=tcp:10.0.2.100:3142-unix:$SNAP_COMMON/pkgproxy.sock, and …:3143-unix:…/pkgproxy-lxd-tls.sock when that socket exists (010-qemu.mk:31-39).
The patch
--- a/net/slirp.c
+++ b/net/slirp.c
@@ -1153,6 +1153,12 @@
"forwarding rule '%s'", config_str);
return -1;
}
+ } else if (g_str_has_prefix(p, "unix:")) {
+ if (slirp_add_unix(s->slirp, &p[5], &server, port) < 0) {
+ error_setg(errp, "Conflicting/invalid host:port in guest "
+ "forwarding rule '%s'", config_str);
+ return -1;
+ }
} else {
Source: guestfwd-unix-socket.patch:22-36.
Which QEMU has it
| Install | QEMU | Patch | pkgproxy reached by |
|---|---|---|---|
| the snap | built in the qemu part, one build for every QEMU component | applied in override-pull (snapcraft.yaml:327-341) | a socket (the default) or a URL |
| the container | built in the build-qemu stage, installed in /usr/local | applied before configure (Dockerfile:28-32) | PKGPROXY_URL |
| native packages, a source install | the distribution's | not applied | PKGPROXY_URL |
Checking a QEMU binary
The smoke test's check, usable by hand: grep -q slirp_add_unix "$(command -v qemu-system-x86_64)" succeeds only for a patched QEMU, since "The patch is the only caller of slirp_add_unix in QEMU" (smoke-pkgproxy.sh:84-90).
Roads from here
PKGPROXY · caching for guests
- How a guest reaches pkgproxy — the socket path and the URL path side by side
- What pkgproxy gives a garden — what pkgproxy gives a garden
SNAP · snap-only features
- The image-garden snap — the snap whose QEMU components carry the patch
CONTAINER · OCI image
- Running image-garden in a container — the container's patched QEMU and PKGPROXY_URL
NATIVE · packages
- Living with the distribution QEMU — what a distribution QEMU means for a native install