Choosing a spread

spread or spread-plus for CI operators

The snap's image-garden.spread app runs spread or spread-plus from their components. A # spread-variant= comment in spread.yaml in the current directory wins over the spread-variant snap setting; the configure hook installs the component the setting names, and the app installs a missing one itself only when run as root, and otherwise says how.

On the map: snap

▶ Watch this deck — 6 scenes, ~5 min

Worth seeing first

01 Two spreads in one snap

TWO SPREADS spread runs your tests in the machines a garden gives it spread the original spread-plus a fork, more features project A project B image-garden.spread one app, two components $ snap alias image-garden.spread spread as the GitHub action does
Two runners, spread and spread-plus, each a component of the snap; one project needs one, another the other; one app, image-garden.spread, runs either; snap alias image-garden.spread spread gives it the plain name

The spread and spread-plus components, one app: image-garden.spread. snap alias image-garden.spread spread gives it the plain name.

spread runs your tests in the machines a garden gives it. And there are two of it. spread is the original, and spread-plus is a fork with more features and fixes. Some projects need one, some the other, and you may work on both on one machine. The snap carries each as its own component, and runs them through one app, image-garden.spread. The GitHub action gives it the plain name spread, with snap alias. This deck shows how the app picks a variant, how its component gets installed, what changes when spread runs inside the snap, and the mistakes that stop it.

Details and sources

Details

  • The components: spread, "Spread integration test runner", and spread-plus, "a fork of Spread with additional features and fixes", each pinned to a Git revision (snapcraft.yaml:46-58). spread-plus comes from github.com/canonical/spread-plus (snapcraft.yaml:564-583).
  • The spread app runs usr/bin/spread-chooser through qemu-var-tmp, sets SPREAD_HOST_PATH: $PATH and plugs kvm, home, network, network-bind, pkgproxy-sockets, docker and docker-executables (snapcraft.yaml:211-226).
  • image-garden-action runs snap alias image-garden.spread spread (action.yaml:394-397).

02 How the app picks

THE CHOOSER $ image-garden.spread … spread-chooser 1. in the current directory ./spread.yaml # spread-variant=plus the first such comment, anywhere in the file 2. else the setting snap get image-garden spread-variant 3. else plain spread exec components/R/spread-plus/spread "$@" with all your arguments the project carries its choice
image-garden.spread runs spread-chooser, which takes the first # spread-variant= comment in ./spread.yaml, else the spread-variant snap setting, else plain spread, and execs the spread in the matching component with all the arguments; so the project carries its choice

# spread-variant=plus in ./spread.yaml, else snap get image-garden spread-variant, else plain spread. The chooser execs components/REVISION/spread-plus/spread with your arguments.

The app runs a small script, spread-chooser, in the directory you start it from. First it looks at spread.yaml there, for a comment that says spread-variant=plus. The first such comment wins, wherever it sits in the file. Without one, it asks the snap setting, spread-variant. Still empty? Then it's plain spread. A variant named plus means the component spread-plus, and the chooser runs the spread in it with all your arguments, as if you'd called it yourself. So a project can carry its choice in its own file, and every machine that runs it gets the same spread.

Details and sources

Details

  • spread-chooser reads spread.yaml in the current directory only, takes the first line matching ^\s*#\s*spread-variant= and strips the prefix and trailing blanks (spread-chooser:5-9).
  • Without a variant from the file it uses snapctl get spread-variant (spread-chooser:51-54).
  • With a variant V it runs the component spread-V, without one the component spread (spread-chooser:56-59), by exec'ing $SNAP/../components/$SNAP_REVISION/COMPONENT/spread "$@" (spread-chooser:38-49); a missing component is the next scene. Both components name their binary spread (snapcraft.yaml:561, snapcraft.yaml:581).
  • The README says the same: the comment goes "on a line of its own anywhere in the file; the first such line wins, and the snap setting applies when there is none" (README.md:181-183). The 0.5.0 entry of the snap's news, which said "in the first few lines" (NEWS.snappy:225-231), was narrower than the code.

Read more

03 Getting the component

GETTING THE COMPONENT configure hook install, refresh, every snap set spread-variant empty +spread spread-variant=plus +spread-plus a fresh install already has plain spread spread-chooser snapctl install +spread-plus as root only as your own user: Notice: … requires the component spread-plus snap install image-garden+spread-plus $ sudo snap set image-garden spread-variant=plus the configure hook installs it, as root
The configure hook, run at install, refresh and every snap set, installs +spread when spread-variant is empty and +spread-plus when it is plus; the chooser installs a missing component itself only when run as root; as your user it shows a message with snap install image-garden+spread-plus and exits 1; or run sudo snap set image-garden spread-variant=plus

sudo snap set image-garden spread-variant=plus installs spread-plus through the configure hook. The chooser installs a missing component only as root; otherwise it shows snap install image-garden+spread-plus and exits 1.

Where does the component come from? Mostly from the configure hook. snapd runs it at install, at refresh and after every snap set, and it installs the component the setting names: spread when the setting is empty, spread-plus when it's plus. So a fresh install already has plain spread. If spread.yaml asks for a variant that isn't installed, the chooser installs it itself, but only when it runs as root, because snapd lets only root do that. As your own user, it shows a message instead, with the command to run: snap install image-garden plus spread-plus. Or choose the variant up front: sudo snap set image-garden spread-variant=plus.

Details and sources

Details

  • The configure hook reads snapctl get spread-variant: empty, it runs snapctl install +spread when components/REVISION/spread/spread is not executable; a valid variant V, snapctl install +spread-V when components/REVISION/spread-V/spread is not; both only for a store revision (configure:83-107).
  • The README: the snap installs image-garden+spread "as it is installed: its configure hook runs as root then", and snap set image-garden spread-variant=plus installs another variant (README.md:163-179). Derived, from snapd: the configure hook runs at install, at refresh and after snap set.
  • The chooser, when the chosen component's spread is not executable, runs snapctl install "+COMPONENT" only when id -u is 0, ignoring its failure; if spread is still missing it shows a message headed Notice, "It seems that the component COMPONENT is not installed", "The image-garden snap requires the component COMPONENT in order to run spread.", then "Install the component with the following command: snap install image-garden+COMPONENT", and exits 1 (spread-chooser:11-49). The README says the same (README.md:184-187).
  • Derived, from snapd (overlord/hookstate/ctlcmd/ctlcmd.go): install is not among the snapctl commands a user other than root may run, which is why the chooser does not try it then.
  • image-garden-action sets the variant with snap set when its spread-variant input is set (action.yaml:398-401).

Read more

04 spread runs inside the snap

INSIDE THE SNAP image-garden snap, confined image-garden.spread ~/your-project home only SPREAD_HOST_PATH=the PATH you started with allocate, discard: PATH="$SPREAD_HOST_PATH" image-garden docker docker-executables docker snap $ snap connect image-garden:docker-executables docker:docker-executables
Inside the snap, image-garden.spread reaches your project only under your home directory; it saves your PATH as SPREAD_HOST_PATH so the garden backend's allocate and discard can restore it before calling image-garden; its docker and docker-executables plugs reach the docker snap once connected

Projects under $HOME. SPREAD_HOST_PATH carries your PATH to the backend's hooks. docker and docker-executables plugs for the docker backend.

Remember that spread itself now runs inside the snap, confined like everything else there. That changes three things. Your project has to live under your home directory, because that's the part of your files the app can reach. spread sets PATH to a fixed list of its own, so the app saves the PATH you started it with as SPREAD_HOST_PATH, so a garden backend's allocate and discard can put it back before they call image-garden. And for spread's docker backend, the app plugs docker and the docker snap's executables: connect both if you use it.

Details and sources

Details

  • The app plugs home, and no removable-media (snapcraft.yaml:219-226).
  • "Capture the PATH environment that spread was invoked with as spread internally clobbers and re-sets PATH to a fixed value": SPREAD_HOST_PATH: $PATH (snapcraft.yaml:215-218). image-garden's own spread.yaml restores it in its allocate and discard (spread.yaml:27-35).
  • The docker-executables content plug mounts the docker snap's executables at $SNAP/docker-snap; the docker stub runs $SNAP/docker-snap/bin/docker or prints how to connect image-garden:docker-executables docker:docker-executables and image-garden:docker docker:docker (snapcraft.yaml:267-270, docker:6-35).
  • The app's command chain includes qemu-var-tmp, so its temporary directory follows the same rule as QEMU's (snapcraft.yaml:212-214).

05 Common mistakes

COMMON MISTAKES # spread-variant= elsewhere only ./spread.yaml is read: not .spread.yaml, not a file in another directory spread-variant=P a-z, 0-9, single dashes spread-variant must start with a lowercase ASCII letter and may only use lowercase ASCII letters, digits and single dashes, not at the end spread-variant=pluss snapd finds no spread-pluss component a sideloaded snap $ snap install --dangerous ./image-garden+spread_*.comp
Four mistakes: a comment in .spread.yaml or a run from another directory gives no variant; a setting that is not a valid component name, such as P, fails with spread-variant must start with a lowercase ASCII letter and may only use lowercase ASCII letters, digits and single dashes, not at the end; a typo such as pluss has no spread-pluss component; a sideloaded snap needs the spread .comp file installed with --dangerous

Only ./spread.yaml is read. A setting that is not a valid component name fails snap set. An unknown variant has no component. A sideloaded snap needs snap install --dangerous ./image-garden+spread_*.comp.

A few mistakes come up. The chooser reads only spread.yaml in the directory you start from. A project that calls its file .spread.yaml, or a run from another directory, gets no variant from the comment. The setting must make a valid component name: lowercase letters, digits and single dashes, starting with a letter and not ending with a dash. Anything else, a capital P say, fails snap set with a message that spells out those rules. A variant with no component, say the typo pluss, fails when snapd finds no spread-pluss component. And a snap installed from a file never fetches a spread component: install its .comp file too.

Details and sources

Details

  • Only spread.yaml in the current directory is read (spread-chooser:5-9). Whether spread itself accepts .spread.yaml or looks in other directories is spread's own behaviour, not in this tree.
  • The configure hook refuses a variant that matches *[!a-z0-9-]*, [!a-z]*, *- or *--*, that is anything but lowercase ASCII letters, digits and single dashes, starting with a letter and not ending with a dash: the rules of component names, since the variant names the component spread-VARIANT. It exits 1 with spread-variant must start with a lowercase ASCII letter and may only use lowercase ASCII letters, digits and single dashes, not at the end, and installs nothing (configure:94-99). A one-letter name such as p is accepted.
  • Derived, from snapd: a failing configure hook makes snap set fail, so the setting is not stored.
  • A name that passes but has no component, such as pluss, makes the hook run snapctl install +spread-pluss (configure:100-106), which snapd refuses because the snap has no such component (derived); the hook then fails, and so does snap set.
  • A sideloaded snap installs no component from the configure hook (configure:87, configure:102); the .comp files are built next to the snap (README.md:228-232).

06 Where to go from here

ROADS image-garden.spread Testing with spread the garden backend allocate and discard SPREAD_HOST_PATH in use The GitHub action its spread-variant input Snap components spread on GitLab in a container
Roads from choosing a spread: testing with spread, the allocate and discard hooks, the GitHub action, the components and spread on GitLab

Next: testing with spread, the allocate and discard hooks, the GitHub action, the components and spread on GitLab.

So the snap runs one of two spreads: the comment in your spread.yaml decides, then the setting, then plain spread, and the configure hook installs what the setting names. Next, the spread deck shows what spread does with a garden backend. The hooks deck shows allocate and discard, and how they use SPREAD_HOST_PATH. On GitHub, the action's spread-variant input sets the same snap setting. The components deck shows the other nine. And on GitLab, spread and spread-plus come in a container instead.

Details and sources

Read more

Basis of the statements on this page

Read in the image-garden source at the commit this map describes ("code"): spread-chooser/spread-chooser, snap/hooks/configure and snap/snapcraft.yaml. When snapd runs the configure hook and who may run snapctl install come from snapd's source ("derived"). Nothing was run for this page.

Which spread runs

OrderSourceExampleComponent
1the first # spread-variant=V line in ./spread.yaml# spread-variant=plusspread-plus
2the snap settingsnap set image-garden spread-variant=plusspread-plus
3neitherspread

The chooser execs $SNAP/../components/$SNAP_REVISION/spread[-V]/spread with all its arguments (spread-chooser:5-59). The README documents the order (README.md:163-187).

Who installs the component

WhoWhenWhat
the configure hookinstall, refresh, every snap set (derived)+spread with the setting empty, +spread-V with spread-variant=V, if missing; not for a sideloaded revision (configure:83-107)
spread-choosereach run as root, if the chosen component is missingsnapctl install +spread[-V], which snapd allows only for root; as any other user it does not try, and shows how to install it (spread-chooser:35-49)
youany timesudo snap install image-garden+spread-plus, or snap install --dangerous with the .comp file for a sideloaded snap

Messages

WhenMessage
a spread-variant that is not a valid component name (lowercase letters, digits and single dashes, starting with a letter, not ending with a dash)spread-variant must start with a lowercase ASCII letter and may only use lowercase ASCII letters, digits and single dashes, not at the end (configure:94-99)
the configure hook installingInstalling component with spread... or Installing component with spread variant (spread-V)... (configure:88, configure:103)
the chosen component missing, and not installed by the choosera message headed Notice: It seems that the component spread-plus is not installed. and The image-garden snap requires the component spread-plus in order to run spread., then Install the component with the following command: and snap install image-garden+spread-plus; exit 1 (spread-chooser:11-33)

Roads from here

SPREAD · tests

BUILT ON IT · CI integrations

SNAP · snap-only features