Validation sets: snaps that belong together

purpose for administrators for Ubuntu Core developers for app developers

Each snap refreshes on its own channel so the combination of revisions on a device may be one nobody tested. A validation set is an account's signed statement that certain snaps and revisions belong together. A device can watch a set or enforce it and a model can enforce one from first boot. Enforcing pins revisions and blocks removal so it trades freedom on the device for a combination that was tested.

On the map: assertions district

▶ Watch this deck — 7 scenes, ~5 min

01 Tested together, refreshed apart

VALIDATION SETS acme's test benchacme-agent 41acme-updater 17acme-vpn 8✓ tested together gateway 600, in the fieldacme-agent 41acme-updater 17acme-vpn 8each snap follows its own channel stable movesacme-agent 42acme-agent 4242 + 17 + 8: tested by nobody validation set acme/gateway-setsigned by acmeacme-agent 41acme-updater 17acme-vpn 8a statement by an account:these snaps, and maybe theserevisions, work well together acme-agent 41enforced with --refresh:the agent goes back to 41
Top: acme's test bench with acme-agent 41, acme-updater 17 and acme-vpn 8, tested together. Bottom: gateway 600 in the field with the same three snaps. The stable channel brings acme-agent 42 to the gateway, giving 42, 17 and 8, a mix nobody tested. In between, the validation set acme/gateway-set signed by acme lists agent 41, updater 17 and vpn 8; enforced with --refresh, the agent goes back to 41

A validation set is an account's signed statement that some snaps, optionally at fixed revisions, belong together. The documentation says it helps interdependent snaps "maintain their testing and certification integrity" and "orchestrate their updates". acme-updater and acme-vpn are invented for this map.

On acme's test bench, agent 41, updater 17 and VPN 8 passed every test, together. In the field, each of them follows its own channel. So what's running on gateway number 600 this morning? Snapd refreshes a snap when its channel moves. The agent went to 42, the other two stayed, and that mix was never on the bench. What's missing is a statement. In snapd's words, a validation set is a statement by an account about a set of snaps, and possibly revisions, that work well together. Any account can publish and sign one. Enforce it on the gateway with refresh, and the agent goes back to 41.

02 Three decisions per snap

REQUIRED · OPTIONAL · INVALID acme/gateway-setacme-agentrequiredrev 41acme-diagoptionalrev 5acme-legacy-vpninvalidgateway acme-agent 41must be installed,at revision 41required is the default acme-diag 5may be installed;if it is, at revision 5 acme-legacy-vpnmust not be installedinvalid takes no revision a revision is optional on every line;with no revision given, any revision satisfies the set
The set acme/gateway-set lists acme-agent required at revision 41, acme-diag optional at revision 5 and acme-legacy-vpn invalid. In the gateway, acme-agent must be installed at 41; acme-diag may be installed and if it is, at 5; acme-legacy-vpn is kept out and must not be installed. A line without a revision is satisfied by any revision

Each snap in a set is required (the default), optional or invalid, with an optional revision. An invalid snap cannot carry a revision.

Think about what acme really wants to say about a gateway. It isn't only revision numbers. A set is a list, and each line is a decision about one snap. Required is the default: the agent has to be installed, and here it has to be revision 41. Optional means the snap is allowed. The diagnostics snap may be there or not, but if it is, it has to be revision 5. Invalid means it must not be on the device at all, like an old VPN snap that acme has retired. The revision is optional on every line. Leave it out, and any revision satisfies the set.

03 Watch first, then constrain

MONITOR · ENFORCE $ snap validate --monitor acme/gateway-setinvalidmonitor: reports only $ snap validateValidation Mode Seq Current Notesacme/gateway-set monitor 3 invalidthe agent is at 42;nothing is blocked $ snap validate --enforce acme/gateway-seterror: cannot apply validation set: cannot enforcevalidation set: validation sets assertions are not met: … $ snap validate --enforce --refresh acme/gateway-setRefreshed/installed snaps "acme-agent" to enforcevalidation set "acme/gateway-set"enforce: constrains seq 2seq 3seq 4a failed step goes backto the previous sequence
A terminal: snap validate --monitor acme/gateway-set prints invalid and snap validate lists the set in monitor mode, sequence 3, invalid, because the agent is at 42. snap validate --enforce fails because the set is not met; with --refresh snapd refreshes acme-agent and enforces the set. A lamp changes from monitor, reports only, to enforce, constrains. A ladder of sequences 2, 3 and 4 shows that a failed step goes back to the previous sequence

--monitor reports whether the device matches a set and blocks nothing. --enforce constrains, and needs the device to satisfy the set first, or --refresh to make it so. A new sequence allows a rollback to the previous one.

You've written the set. Do you dare switch it on across the fleet? Start by watching. snap validate --monitor tracks the gateway-set and tells you, when you ask, whether this device matches. Here it says invalid, because the agent is at 42. Nothing is blocked. Enforce is the mode that constrains, and snapd won't start enforcing a set the device doesn't already satisfy. Add the --refresh and snapd first installs or refreshes what's needed. Sets are published in numbered sequences. The docs give the reason: if applying a new sequence fails, snapd can go back to the previous one. Raise the sequence whenever a production set changes.

04 Enforced from the factory for Ubuntu Core developers for administrators

IN THE MODEL type: modelmodel: acme-gateway…validation-sets:- account-id: acmename: gateway-setmode: enforce image builtfirst bootyears in the fieldfirst boot: the set is enforced at the end of seeding $ snap validate --forget acme/gateway-seterror: validation-set is enforced by the model or mode: prefer-enforceenforced at first boot too, may be forgotten later sequence: 3$ snap validate --enforce acme/gateway-set=4error: … only sequence 3 allowed by modelsequence: only that sequence is allowed the model decides: is the set advice, or part of what the device is?
The acme-gateway model lists validation-sets with account-id acme, name gateway-set and mode enforce. A timeline runs from image built to first boot, where the set is enforced at the end of seeding, to years in the field. snap validate --forget is refused with validation-set is enforced by the model. Mode prefer-enforce is enforced at first and may be forgotten later. sequence 3 in the model refuses any other sequence with only sequence 3 allowed by model

A model lists validation sets with mode enforce or prefer-enforce. Both are enforced from first boot; only a prefer-enforce set can be forgotten. A sequence in the model allows only that sequence.

Typing snap validate works for the gateway on your desk. The other 999 need the same rule before anyone has touched them. So acme writes the set into the gateway's model. Each device then enforces it from its first boot, at the end of seeding. Mode enforce holds for as long as the model lists the set, and snap validate --forget is refused. prefer-enforce starts the same way, but the set may be forgotten later. The model can also fix a sequence, and then only that sequence is allowed. So the model answers a question the set can't: is this statement advice, or part of what the device is?

05 What enforcing costs

THE COST $ snap remove acme-vpnerror: cannot remove "acme-vpn": snap "acme-vpn" is not removable:snap "acme-vpn" at revision 8 is required by validation sets: 16/acme/gateway-set/3 $ snap refresh acme-agent --revision 42cannot update snap "acme-agent" to revision 42 without --ignore-validation,revision 41 is required by validation sets: 16/acme/gateway-set/3 $ snap refresh acme-agent --revision 42 --ignore-validationallowed, and remembered for acme-agent on later refreshes $ snap validateValidation Mode Seq Current Notesacme/gateway-set enforce 3 invalid a snap file installed by hand is not checkedduring a multi-snap update, old and new revisions can coexist briefly: not a tool for hard version dependenciesacme stands in for the account id, a 32-character string
A terminal with acme/gateway-set enforced. snap remove acme-vpn fails because acme-vpn at revision 8 is required by validation sets 16/acme/gateway-set/3. snap refresh acme-agent --revision 42 fails without --ignore-validation because revision 41 is required. With --ignore-validation it is allowed and remembered for acme-agent, and snap validate then lists the set as enforce, sequence 3, invalid. Below: a snap file installed by hand is not checked, and during a multi-snap update old and new revisions can coexist briefly

Under an enforced set, required snaps cannot be removed and other revisions are refused. --ignore-validation overrides it for one snap and is remembered. Local installs are not checked.

A statement that binds has to cost something. Here's what you give up on that gateway. Removing the VPN fails, because the set requires it, and the error names the set. Asking for agent 42 is refused too, since revision 41 is required. There is a way out: --ignore-validation. It isn't a one-off. Snapd remembers it for that snap on later refreshes, and the set now reports invalid. The docs are open about two limits. A snap file you install by hand isn't checked. And while several snaps update, old and new revisions can coexist for a moment, so a set is no tool for hard version dependencies.

06 When another tool fits better

WHEN NOT one snap misbehaves on this machine$ snap refresh --hold acme-agenta hold some devices stay on an older line1.x/stablea track these snaps, at these revisions, on every devicevalidation seta combination,across a fleet approve each new revision of another snapgating snap + validation assertionolder: refresh controlin the Ubuntu Core docs the code: "not to be confused with validation sets"snapd still applies that gating when it refreshes an installed snap, unless --ignore-validation is given
Four needs and the tool for each. One snap misbehaves on this machine: snap refresh --hold acme-agent, a hold. Some devices stay on the old major version: a track such as 1.x/stable. These snaps at these revisions on every device: a validation set. Approve each new revision of someone else's snap: a gating snap and the validation assertion, the older refresh control mechanism that snapd still applies when it refreshes an installed snap

Use a hold for one snap on one machine and a track for an older line of releases. A validation set is for a combination across devices. The older gating-snap mechanism, the validation assertion, still gates refreshes.

Before you publish a set, ask whether the problem is really a combination. Often it's smaller. If one snap misbehaves on one machine, hold that snap. If some devices should stay on an older line of releases, that's what tracks are for. Reach for a set when the unit is a combination and the audience is a fleet. You may also meet an older mechanism in the Ubuntu Core docs: refresh control with a gating snap. Its publisher approves each revision of another snap with a validation assertion. The code warns readers not to confuse the two. Snapd still applies that gating when it refreshes an installed snap.

07 Where to go from here

ROADS Validation sets Assertions: snaps and validationthe assertion, modes, refusals Assertions: how they workhow any assertion is checked Who decides when a snap updatesholds, timers and sets Refreshthe change a set steers Model, gadget and kernela model that lists setsubuntu-seed and first bootsets enforced when seeding ends Recovery systemssets in a new recovery systemA store of your ownwhere a brand publishes sets
Roads from validation sets: Assertions: snaps and validation; Assertions: how they work; Who decides when a snap updates; Refresh; Model, gadget and kernel; ubuntu-seed and first boot; Recovery systems; A store of your own

The assertion and its modes are in "Assertions: snaps and validation"; the other ways to control updates are in "Who decides when a snap updates".

So that's what a validation set is for: a signed statement that keeps a tested combination together, watched or enforced. The format and the exact refusals are one deck over, with the snap assertions. Assertions: how they work shows how any assertion is signed and checked. Who decides when a snap updates compares holds, timers and sets. Refresh is the change that a set steers. The model and first boot decks show sets written into a model and enforced at the end of seeding. And recovery systems and a store of your own cover the rest.

Read more

What the sources say a validation set is for

  • snapd: "a statement by an account about a set snaps and possibly revisions for which an extrinsic/implied property is valid (e.g. they work well together)" (asserts/validation_set.go, quoted as written)
  • the documentation: to help a group of interdependent snaps "maintain their testing and certification integrity, as well as help orchestrate their updates", to simplify dependency deployment, and to help manage devices

A set is signed by the account it belongs to: its authority-id must equal its account-id.

Presence

presenceMeaningWith a revision
required (default)the snap must be installedit must be at that revision
optionalthe snap may be installedif installed it must be at that revision
invalidthe snap must not be installeda revision is an error here

Modes on a device

ModeSet byEffect
monitorsnap validate --monitor <account>/<name>[=<seq>]prints valid or invalid; blocks nothing
enforcesnap validate --enforce [--refresh] <account>/<name>[=<seq>]constrains install, refresh and remove; refused unless the device already satisfies the set, or --refresh makes it so
enforce (model)model validation-sets with mode: enforceenforced from first boot, at the end of seeding; cannot be forgotten
prefer-enforce (model)model validation-sets with mode: prefer-enforceenforced from first boot; can be forgotten

snap validate --forget <account>/<name> stops tracking a set. =<seq> pins a sequence; without it the set may move to newer sequences. A model entry without account-id uses the model's brand. A model entry with sequence allows only that sequence: only sequence <n> allowed by model.

--refresh installs and refreshes snaps to satisfy the set. It does not remove snaps the set marks invalid.

What an enforced set changes

OperationResult
install a snap marked invalidcannot install snap "<snap>" due to enforcing rules of validation set <key>
refresh to another revisioncannot update snap "<snap>" to revision <n> without --ignore-validation, revision <m> is required by validation sets: <keys>
remove a required snapsnap "<snap>" is required by validation sets: <keys>, or snap "<snap>" at revision <n> is required by validation sets: <keys> when the set gives a revision
refresh alleach snap is steered to the revision the sets require; for a pinned revision the channel and cohort are not used
install a local filenot checked
--ignore-validationallowed; remembered for that snap on later refreshes; the set then lists as invalid
forget a set the model enforcesvalidation-set is enforced by the model

A set's key is printed as 16/<account-id>/<name>/<sequence>. Parallel instances are not constrained.

Validation sets and gating snaps

Validation setGating snap (refresh control)
Assertionvalidation-setvalidation plus refresh-control in the gating snap's snap-declaration
Signed byany account for itselfthe gating snap's publisher
Sayswhich snaps and revisions belong togetherthis revision of that snap is approved
Applies whenthe device tracks the set or the model lists itthe gating snap is installed and an installed gated snap refreshes
Set up bythe account alone with snapcraft edit-validation-setsthe store team sets refresh-control first

--ignore-validation skips both.

Further reading

Roads from here

GETTING SNAPS

RUNNING SNAPS

ASSERTIONS · signed statements snapd trusts

UBUNTU CORE · install and recovery