Epochs: when new code cannot read old data

purpose for app developers for administrators

A refresh replaces a snap's code and keeps its data so a release with a new data format could leave users with data it cannot read. An epoch states which data formats a revision can read and write. snapd and the store only move a device to a revision that can read what the current one wrote and a publisher can ship a migrating revision as the step in between. Most snaps stay at epoch 0 and need none of this.

On the map: getting snaps district

▶ Watch this deck — 7 scenes, ~5 min

01 New code, old data

EPOCHS photo-editor 2.4 · rev 42reads format Ayour data: catalogue, format A refreshphoto-editor 3.0 · rev 44the code is replaced, the data is kept reads format Byour catalogue: unreadable epoch 0epoch 1written by epoch 0 snapd does not make this jump
photo-editor 2.4 revision 42 sits on top of your data, a catalogue in format A. A refresh replaces it with photo-editor 3.0 revision 44 and keeps the data as it was; 3.0 reads format B, so the catalogue is unreadable. Revision 42 is tagged epoch 0 and revision 44 epoch 1, the data was written by epoch 0, and snapd does not make this jump

"An Epoch represents the ability of the snap to read and write its data." snapd only moves a device to a revision that can read what its current revision wrote. The photo editor's versions and epochs in this deck are invented for the example.

The photo editor's next version has a faster catalogue. It also can't read the old one, and you have six years of photos in the old one. What should happen tonight, when the refresh runs? A refresh replaces the program and leaves your data where it is. Here, it would hand your catalogue to code that can't open it. So the snap states which data it can read. That statement is the epoch. In the code's words, it's the ability of the snap to read and write its data. And snapd won't make the jump to a revision that can't read what the current one wrote.

02 Most snaps need none of this

EPOCH 0 nobody wrote this: it is the defaultrev 38epoch 0rev 39epoch 0rev 40epoch 0rev 41epoch 0rev 42epoch 0 every refresh is allowed: 0 reads 0 format changed, but the new code still reads the old filesepoch 0no bump needed rev 44epoch 1only when new code cannot read old data
Five revisions, rev 38 to rev 42, all at epoch 0, which nobody wrote because it is the default; every refresh between them is allowed because 0 reads 0. A format change that the new code still reads keeps epoch 0 with no bump. Only rev 44, dimmed, has epoch 1, for when the new code cannot read the old data

Snaps default to epoch 0, and "most developers need not worry about it". The epoch changes only when new code cannot read old data.

If you publish a snap and you've never typed the word epoch, relax. You've been using one all along. Every snap has an epoch, and the default is zero. The comment in the code says it plainly: most developers need not worry about it. Zero reads zero, so every refresh between your revisions is allowed. You don't bump the epoch for a new version. You don't even bump it for a new file format, as long as the new code still reads the old files. You bump it only at a real break, when the new code can't read the old data.

03 The step in between for app developers

EPOCH: 1* rev 42 · 2.4nothing: epoch 0reads 0writes 0 rev 44 · 3.0epoch: 1reads 1writes 11 does not read 0 rev 43 · 2.9epoch: 1*reads 0, 1writes 1 reads what42 wrotereads what43 wrote this revision converts the catalogue.snapd converts nothing. more than one step back:epoch: {read: [...], write: [...]}
Three revisions with their snap.yaml epoch line: rev 42 version 2.4 has no line, epoch 0, and reads and writes 0; rev 44 version 3.0 has epoch: 1, reads and writes 1, and the direct jump from 42 is cut because 1 does not read 0; rev 43 version 2.9 has epoch: 1*, reads 0 and 1 and writes 1. Arrows 42 to 43 and 43 to 44 make a path. Revision 43 converts the catalogue; snapd converts nothing. A long form lists read and write epochs explicitly

epoch: N reads and writes N. epoch: N* also reads N-1, so it can migrate. The snap converts its own data; snapd only checks that the revision can read what the installed one wrote.

You're the developer now, and you really do need that new catalogue format. Version 3.0 says epoch 1 in its snap.yaml, and epoch 1 can't read zero. Ship the step in between first: version 2.9, revision 43, with epoch: 1*. The star means it can also read the epoch before it. So it reads zero and one, and writes one. Now there's a path: 42 to 43, then 43 to 44. And who does the work? Converting the catalogue is revision 43's job, in its own code or its hooks. Snapd only compares the numbers. For anything more elaborate, there's a long form with read and write lists.

04 One refresh, two revisions

WHAT YOU SEE 42epoch 043epoch 1*44epoch 1 $ snap refresh photo-editor storeI have 42, epoch 0then take 43 again, for photo-editor?44photo-editor 3.0 from Example Studio refreshedsame change, no second command $ snap tasks --last=refreshDownload snap "photo-editor" (43) ......Download snap "photo-editor" (44) ...two downloads, one refresh a first install: no old data to read,so the store can offer 44 directly
A track with three stops: 42 at epoch 0, 43 at epoch 1*, 44 at epoch 1. snap refresh photo-editor tells the store I have 42, epoch 0, and the store answers take 43; the same change asks again for photo-editor and the store answers 44. snap tasks --last=refresh shows two downloads, revision 43 and revision 44. A first install has no old data and can get 44 directly

A refresh across an epoch change goes through the migrating revision and on to the latest in the same change. A first install sends no installed epoch, so the store can offer the latest revision directly.

So what does all this look like from your side of the screen? One command. You type snap refresh photo-editor, or the timer does. Snapd tells the store which revision you have, and its epoch. The store offers a revision that can read your data: 43, not the newest. 43 is installed, and its migration runs. Then the change asks the store once more, about the snaps it just refreshed. Your data is epoch 1 now, so the answer is 44. Look at the task list afterwards: two downloads, one refresh. A first install has no old data to read, so the store can offer 44 straight away.

05 When snapd says no

THE REFUSAL what is installedrev 42 · epoch 0 $ snap install --dangerous photo-editor_3.0_amd64.snapcannot refresh "photo-editor" to local snap with epoch 1,because it can't read the current epoch of 0 a store revision that skips the step in between:cannot refresh "photo-editor" to new revision 44 with epoch 1,because it can't read the current epoch of 0 $ snap restore 7 photo-editorcannot restore snapshot for "photo-editor": current snap (epoch 1)cannot read snapshot data (epoch 0)snapshot 7: epoch 0rev 44 · epoch 1 in all three, the data is left as it was
Three refusals. snap install --dangerous of photo-editor 3.0 over data at epoch 0: cannot refresh to local snap with epoch 1 because it can't read the current epoch of 0. A store revision 44 that skips the bridge: cannot refresh to new revision 44 with epoch 1, same reason. snap restore 7 photo-editor with snapshot 7 at epoch 0 and rev 44 at epoch 1 installed: current snap (epoch 1) cannot read snapshot data (epoch 0). The data is untouched in all three

snapd refuses a local install or a store refresh whose epoch cannot read the installed one, and a snapshot restore that the installed revision cannot read. The message names both epochs.

Mostly the store routes you around the gap. Here's what happens when something jumps it anyway. Say you build 3.0 yourself and install the file over 2.4. Snapd refuses, and the message names both epochs. The test for this states why: an update mustn't leave the snap unable to read the previous version's data. The same check runs on a revision that comes from the store. And it runs in one place you might not expect: restoring a snapshot. A snapshot remembers the epoch it was taken under. If the installed revision can't read it, the restore is refused. In all three cases, your data is left as it was.

06 Epoch, track, or revert

WHICH TOOL epoch424344everyone crosses, through the step track2.x/stablelatest/stablesome stay on the old linefor as long as it is published revert4243$ snap revert photo-editor42 gets back the data it had before the refresh data shared by all revisions is not rolled back an epoch does not convert data, keep the old version alive, or undo anything
Three lanes. Epoch: 42, 43 and 44 on one line with one arrow through them, everyone crosses through the step in between. Track: two lines, 2.x/stable and latest/stable, with a device staying on the old line for as long as it is published. Revert: an arrow back from 43 to 42 with snap revert photo-editor, 42 gets back its own data, but data shared by all revisions is not rolled back. An epoch does not convert data, keep the old version alive or undo anything

An epoch orders a migration for everyone on a channel. A track keeps an older line of releases available. snap revert returns to the previous revision and its own data, but not data shared by all revisions.

Three tools live close together, and people reach for the wrong one. Ask yourself where you want your users to end up. An epoch is for a break that everyone should cross. It makes sure they cross in the right order, and it doesn't keep the old version around. If some people should stay on version two, that's a track: a second line of releases the publisher keeps side by side. If the new version went wrong, snap revert photo-editor puts the previous revision back, with its own copy of the data. One caveat: data shared by all revisions isn't rolled back. The epoch converts nothing and undoes nothing.

07 Where to go from here

ROADS Epochsthe purpose Snap packagingwhere each revision keeps its data Refreshthe task that asks the store again The storetracks, what a refresh request says Snapshotsthe epoch check on restore Remove and revertwhat a revert does not undo Update controlwho else can make an update wait
Roads from Epochs: snap packaging for where each revision keeps its data, refresh for the task that asks the store again, the store for tracks and refresh requests, snapshots for the epoch check on restore, remove and revert for what a revert does not undo, and update control for who else can make an update wait

Roads: snap packaging, refresh, the store, snapshots, remove and revert, and who decides when a snap updates.

That's the whole idea: a number that stops new code from meeting data it can't read. Zero by default, and a starred step in between when the format breaks. The machinery around it lives in other decks. Snap packaging shows where each revision keeps its data. The refresh deck has the task that asks the store again. The store deck covers tracks, and what a refresh request says. Snapshots has the epoch check on restore. Remove and revert covers what a revert doesn't undo. And the update control deck lists everyone else who can make an update wait.

Read more

The ways to write an epoch

In snap.yamlReadsWritesUse
nothing, or epoch: 000the default
epoch: NNNafter the break
epoch: N*N-1 and NNthe migrating revision
epoch: {read: [1, 2, 3], write: [1, 3]}the read listthe write listanything else

Rules for the long form: at most 10 entries per list, strictly increasing, and the two lists must share at least one number. read defaults to write; write defaults to the last entry of read. Numbers are written in base 10 with no zero padding and must be less than 2³². 0* is invalid.

The rule

A revision may replace the installed one only if its read list and the installed revision's write list have a number in common.

InstalledOfferedAllowed
00yes
01no
01*yes
1*1yes
10no
12*yes

Where the epoch is checked

MomentWho checksResult if unreadable
choosing a revision to offerthe store, from the epoch snapd reportsa readable revision is offered instead
refresh from the storesnapdcannot refresh "<snap>" to new revision <n> with epoch <e>, because it can't read the current epoch of <e>
snap install --dangerous or snap try over an installed snapsnapdcannot refresh "<snap>" to local snap with epoch <e>, because it can't read the current epoch of <e>
end of a refresh changesnapd asks the store again for the snaps it refresheda revision with a different epoch is added to the same change
snap restoresnapdcannot restore snapshot for "<snap>": current snap (epoch <e>) cannot read snapshot data (epoch <e>)
snap revertno epoch check was found-

Epoch, track and revert

ToolDecided byEffect
epochthe publisher, in snap.yamldevices move across a data format change in a readable order
trackthe publisher, in the storean older line of releases stays available to those who follow it
snap revertthe administratorthe previous revision and its own data return; data shared by all revisions is not rolled back

Further reading

Roads from here

GETTING SNAPS

  • Snap packaging — where a revision's data lives and the default epoch in snap.yaml
  • The store — tracks and what a refresh request tells the store

RUNNING SNAPS