snapctl: services, refresh and changes

snapctl command group for users for app developers for Ubuntu Core developers for snapd developers for administrators

Eleven snapctl commands a snap uses on itself only. It starts, stops and restarts its own services, installs and removes its own components, watches the changes it started, reads its refresh status and, from the gadget install-device hook, decides how install mode ends.

On the map: snapctl district

▶ Watch this deck — 10 scenes, ~7 min

01 The snap acts on itself for app developers for Ubuntu Core developers for snapd developers for administrators

THE SNAP ACTS ON ITSELF acme-agenthookappsnapctlsnapd identity = the calling snapno snap name to choose service controlstart stop restartservice-control tasks component managementinstall removeonly this snap's components queries and one state writeservices refresh is-readychange tasks reboot $ snapctl install other-snap+extraerror: snapctl: cannot install snaps using snapctl
acme-agent's hook and app call snapctl into snapd, which acts only on the calling snap in three shapes: service control, component management, and queries with one state write; naming another snap is refused

These commands act on the calling snap only: its services, its components, its own changes and its refresh status. reboot works only in the gadget's install-device hook, and refresh only from apps. A snap name other than the caller is refused.

acme-agent wants to restart its own service after a settings change, or fetch an optional component. Could it restart some other snap's service too? No. These eleven commands act only on the snap that calls them. snapd knows which snap that is from the cookie, and there's no argument to name another one. Start, stop and restart control the snap's own services. Install and remove handle only its own components. The rest read state: service status, refresh status, and the changes the snap started, while reboot records one choice on a task. Ask to install another snap, and snapctl refuses.

02 Same command, three timings for app developers for Ubuntu Core developers for snapd developers

THREE TIMINGS $ snapctl restart acme-agent.agentcalled from three hooks of a first install link-snap run-hookinstall run-hookdefault-configure start-snap-services run-hookconfigure run-hookcheck-health first install only run-hookconfigure service-control mark-seededset-model final tasks wait for it run-hookdefault-configure service-control right after start-snap-services run-hookinstall new change: service-controlRunning service command for snap "acme-agent" snapctl waits, up to 2.5 min by default a configure hook cannot observe its own restart
The tasks of a first install, with snapctl restart called from three hooks: from configure the service-control task is appended after check-health and before mark-seeded or set-model; from default-configure it is injected after start-snap-services; from install a separate service-control change runs and snapctl waits for it

From configure the service tasks join the hook's change and run after it; from default-configure they run right after start-snap-services; from other hooks and apps a new service-control change runs and snapctl waits for it, up to 2.5 minutes by default.

acme-agent calls snapctl restart for its agent service. When does the restart actually happen? That depends on the hook. From the configure hook, the service-control task joins the hook's own change, after every task in its lane and before mark-seeded or set-model. So the restart comes after the hook ends. From default-configure, which runs only on first install, it's injected right after start-snap-services. From any other hook, such as install, or from an app, snapd makes a new change and snapctl waits for it, up to half the configure hook timeout. So a configure hook can't check its own restart.

03 Disabled by the snap, kept by snapd general version

KEPT DISABLED meta/hooks/installsnapctl stop --disable acme-agent.uploader# the uploader stays off until set up run-hook[install] separate change service-controlstop + disable snap statedisabled by hooks: uploader start-snap-servicesskips services disabled by hooks on refresh stop-snap-servicesrecords the disabled units inthe disabled list start-snap-servicesdisables them again snapctl start --enableoverrides install-mode: disable
An install hook runs snapctl stop --disable; a separate service-control change stops and disables the services and snapd records them as disabled by hooks; start-snap-services skips them; on refresh stop-snap-services records the disabled units and start-snap-services disables them again; snapctl start --enable overrides install-mode: disable

snapctl stop --disable in the install hook ships a service off by default. snapd records it and keeps it disabled across refreshes; snapctl start --enable overrides install-mode: disable.

acme-agent ships an uploader that should stay off until someone sets it up. How does it stay off? Its install hook calls snapctl stop --disable for the uploader. The install hook isn't one of the queued ones, so a separate service-control change stops and disables the unit while the install waits. snapd records the name as disabled by hooks, and start-snap-services leaves it alone. On refresh, stop-snap-services notes which units are disabled, and start-snap-services disables them again. snapctl start --enable is the mirror image, and it even overrides install-mode: disable in snap.yaml.

04 A service the snap switched off for users for administrators in place of the general version

SWITCHED OFF BY THE SNAP $ sudo snap install acme-agent $ snap services acme-agentService Startup Current Notesacme-agent.agent enabled active -acme-agent.uploader disabled inactive - the snap chose thiswhile it was being installed $ sudo snap refresh acme-agent$ snap services acme-agentService Startup Current Notesacme-agent.agent enabled active -acme-agent.uploader disabled inactive -a refresh keeps it off $ sudo snap start --enable acme-agent.uploaderyou decide to run it
After snap install, snap services shows the uploader service disabled and inactive; the snap chose this; after snap refresh they are still disabled; snap start --enable runs one

A snap can install a service disabled on purpose. It stays disabled across refreshes; use snap start --enable to run it.

You install acme-agent on the gateway, check its services, and the uploader is disabled and inactive. Is something broken? No. The snap switched it off itself while it was being installed, on purpose. And a refresh won't switch it back on: after snap refresh, the uploader is still disabled and inactive, because snapd remembers the snap's choice. When you're ready to use it, start it yourself with snap start --enable so that it also starts at boot.

05 Components on request for app developers for Ubuntu Core developers for snapd developers

COMPONENTS ON REQUEST app · snapctl client snapd /v2/snapctl $ snapctl install +modbusX-Snapctl-Features: async change 42 new change kind: snapctl-installinitiated-by-snap:acme-agent change-id: 42returned at once snapctl is-ready 42, every 100 msexit 3 not ready, ask againexit 0 done, snapctl exits 0exit 2 failed, error from stderr with --no-wait the ID is printed instead from a hookrun-hook[configure]→ download-component … link-componentqueued into the hook's change; runs after it change ID 42 and the component are examples
An app runs snapctl install +modbus with the async feature header; snapd creates a snapctl-install change marked initiated-by-snap and returns its ID; the client polls is-ready every 100 ms until exit 0 or 2; from a hook the component tasks are queued into the hook's change and run after it

From an app, snapctl install and remove create a change the snap owns and the client polls is-ready; --no-wait prints the change ID. From a hook they are queued and run after the hook. The component name and change ID are examples.

Say acme-agent finds a Modbus device and wants its optional modbus component. What does the app do while it installs? snapctl install handles only the calling snap's components, from the store, for its current revision. From an app, the client tells snapd it can do the waiting. snapd creates a snapctl install change, marks it as started by this snap, and returns the change ID at once. The client then asks snapctl is-ready every hundred milliseconds until it hears done or failed. With --no-wait, it prints the ID instead. From a hook, the tasks join the hook's change and run after it. Remove works the same way.

06 A change only its snap can see for app developers for snapd developers

ONLY ITS OWN CHANGES $ snapctl remove +modbus --no-wait42 $ snapctl is-ready 42; echo $?3 is-ready exit codes0 Done1 not found or invalid2 Undone, Error or Hold3 not ready $ snapctl tasks 42Status Spawn Ready SummaryDoing today at 10:02 UTC - Run remove hook of …$ snapctl is-ready 42; echo $?0 42 snapctl-removeinitiated-by-snap: acme-agent 17 service-controlnot initiated by the snap $ snapctl is-ready 17error: snapctl: change "17" not found one answer per change every 100 ms;faster callers wait change IDs and times are examples
Inside the app: snapctl remove +modbus --no-wait prints change 42; is-ready exits 3 then 0; tasks prints the task table; the exit codes 0 to 3; change 42 is initiated by the snap, change 17 is a service-control change and is reported as not found

is-ready, change and tasks see only changes the snap started with snapctl install or remove from an app. Anything else is change "ID" not found. Change IDs and times are examples.

Now acme-agent removes that component with --no-wait, and gets a change ID back. How does it follow along? snapctl is-ready answers with an exit code only: zero for done, three for not ready, two for finished without success, and one for anything it can't see. Tasks, also spelled change, prints the task table and exits zero whatever the state. Both see only changes marked as started by this snap, which today means install and remove from an app. A service-control change, or anything you started yourself, is reported as not found. And snapd answers about one change at most every hundred milliseconds; faster callers wait.

07 snapctl refresh: what used to exist and what remains for app developers for administrators for snapd developers

WHAT REMAINS OF REFRESH used to existrun-hook[gate-auto-refresh]$ snapctl refresh --pending --hold$ snapctl refresh --pending --proceed the hook no longer runs; holds placed bysnaps are removed when snapd starts $ snapctl refresh --proceederror: snapctl: cannot proceed:gate-auto-refresh-hook is disabledfrom an app with snap-refresh-control connected remains$ snapctl refresh --pendingpending: readychannel: stableversion: 2revision: 2base: falserestart: false $ snapctl refresh --trackingchannel: latest/stable--show-lockprints the run inhibition hint $ sudo snap refresh --hold acme-agentthe user's hold, holder "system": kept
Left, struck through: the gate-auto-refresh hook with snapctl refresh --hold and --proceed, which now fail; holds placed by snaps are removed. Right: snapctl refresh --pending, --tracking and --show-lock still work, and the user's snap refresh --hold is kept

snapctl refresh --hold and --proceed no longer work: the gate-auto-refresh feature is permanently disabled. --pending, --tracking and --show-lock remain, and a hold set with snap refresh --hold is kept.

You may find old guides where a snap holds back its own refresh. Can acme-agent still do that? No. Refresh is a hidden snapctl command, and it used to let a snap hold or release pending refreshes from the gate-auto-refresh hook. That feature is now permanently disabled: the hook doesn't run, hold and proceed fail, and holds placed by snaps are removed when snapd starts. What remains only reads state. Pending says whether a refresh is ready, inhibited or none. Tracking shows the channel, and works without root. --show-lock prints the run inhibition hint. And your own snap refresh --hold is kept.

08 How install mode ends for Ubuntu Core developers for snapd developers

HOW INSTALL MODE ENDS install-system change setup-run-system setup-ubuntu-save run-hookinstall-device restart-system-to-run-mode gadget install-device hook#!/bin/bashsnapctl reboot --poweroff reboot: {"op": "poweroff"} reboot to run modedefault power off the factory-reset change uses the same hook and task; from any other hook:error: snapctl: cannot use reboot command outside of gadget install-device hook
The install-system change: setup-run-system, setup-ubuntu-save, the gadget install-device hook, restart-system-to-run-mode; the hook runs snapctl reboot --poweroff, which stores a reboot option on the restart task; the device powers off instead of rebooting to run mode

Only the gadget's install-device hook can call snapctl reboot --halt or --poweroff. The choice is stored on restart-system-to-run-mode and applied when install mode or factory reset ends.

The acme-gateway is being installed on the factory floor. When install mode finishes, should it reboot into run mode, or power off and wait for the box? The gadget decides. The install-system change ends with a task called restart-system-to-run-mode, and when the gadget has an install-device hook, it runs just before that task. In the hook, snapctl reboot with halt or poweroff doesn't act at once: it records the choice on the restart task. When that task finishes, the gateway halts or powers off instead of rebooting. A factory-reset uses the same hook and task. Any other hook is refused.

09 Where to go from here

ROADS snapctl: services, refresh and changes snapctl: how it workshow a call finds its context Serviceshow snapd runs snap services Hookswhere these commands run The overlordchanges, tasks and waiting Refreshrefresh and the holds that remain Boot and recoveryinstall mode and run mode snapctl: configuration and healththe configure hook's other half
Roads from snapctl services, refresh and changes: how snapctl works, services, hooks, the overlord, refresh, boot and recovery, and snapctl configuration and health

Roads: how snapctl works, services, hooks, the overlord, refresh, boot and recovery, and snapctl configuration and health.

Services, components, changes, a retired refresh hold, and the gadget's say in how install mode ends. Here's where the surrounding pieces live. The snapctl deck explains how a call reaches snapd and finds its context. Services covers how snapd runs and tracks the agent's daemons. Hooks shows where these commands are called from, and the overlord covers changes, tasks and how they wait for each other. Refresh covers refresh itself and the holds that remain. Boot and recovery covers install mode and the reboot into run mode. And the configuration and health deck covers the configure hook and snapctl get and set.

10 Every command at a glance

EVERY COMMAND AT A GLANCE commandwherewhoeffect changefrom apps and hooksnon-root allowedreads only installfrom apps and hooksroot onlycreates or joins a change is-readyfrom apps and hooksnon-root allowedreads only rebootonly in the install-device hookroot onlychanges state directly refreshfrom apps onlynon-root allowedreads only removefrom apps and hooksroot onlycreates or joins a change restartfrom apps and hooksroot onlycreates or joins a change servicesfrom apps and hooksnon-root allowedreads only startfrom apps and hooksroot onlycreates or joins a change stopfrom apps and hooksroot onlycreates or joins a change tasksfrom apps and hooksnon-root allowedreads only
Every snapctl command in this group with where it may run, who may run it, and what it does to state

11 commands: 5 non-root, 5 create or join a change, 0 saved when the hook or call ends, 1 hook-specific.

All 11 commands of this group at a glance. 5 may run as non-root: change, is-ready, refresh, services and one other. 5 create a change or add tasks to the running one: install, remove, restart, start and one other. One works only inside one particular hook: reboot in install-device. The refresh command now works only from apps. The rest read state or change it directly. Each command is described in full in the list under this deck.

Read more

The catalogue below this deck describes each command. This page collects where their work runs and what a snap can see of it.

Where the work runs

CommandFrom configureFrom default-configureFrom another hookFrom an app or service
start, stop, restarttasks join the hook's change, after the hooktasks injected right after start-snap-servicesa new service-control change; snapctl waitsa new service-control change; snapctl waits
install, remove (components)tasks join the hook's change, after the hookas configuretasks join the hook's change, after the hooka new snapctl-install or snapctl-remove change owned by the snap
services, refresh, is-ready, change, tasksread onlyread onlyread only (refresh is refused in hooks)read only
rebootrefusedrefusedonly install-device: sets an option on a later taskrefused

snapctl waits for a service-control change for up to half the configure hook timeout, 2.5 minutes by default, and then fails with <action> command is taking too long. An older snapctl client without the async feature waits up to 10 minutes for an install or remove.

for app developers for snapd developers

Following a change the snap started

CallOutputExit code
snapctl install --no-wait +<comp>the change ID0
snapctl is-ready <id>nothing0 done; 2 finished as Undone, Error or Hold; 3 not ready; 1 not found or invalid
snapctl tasks <id> or snapctl change <id>the task table, or JSON with --format json0 whatever the state

Only changes with initiated-by-snap equal to the caller are visible; any other ID gives change "<id>" not found. Repeated calls about one change are spaced at least 100 ms apart. --no-wait in a hook fails.

Services kept disabled

ActionRecorded asEffect on refresh
snapctl stop --disable <svc>disabled by hooksthe service stays disabled
snapctl start --enable <svc>enabled by hooksthe service stays enabled, also over install-mode: disable
snap stop --disable or snap start --enable by the userthe unit statea disabled unit is noted before the refresh and disabled again after it

for app developers for administrators for snapd developers

snapctl refresh

FlagWorksOutput
--trackingfrom apps, also without rootchannel: <channel>
--pendingfrom apps, as rootpending (ready, inhibited or none), channel, version, revision, base, restart; cohort with snap-refresh-control
--show-lockfrom apps, as rootthe run inhibition hint, empty when none
--holdno longercannot hold outside of gate-auto-refresh hook from an app
--proceedno longercannot proceed: gate-auto-refresh-hook is disabled

The gate-auto-refresh hook is not run. Holds placed by snaps are removed when snapd starts; holds set with snap refresh --hold (holder system) are kept.

for Ubuntu Core developers for snapd developers

How install mode ends

ChangeTasks around the hook
install-systemsetup-run-system, setup-ubuntu-save, run-hook (install-device), restart-system-to-run-mode
factory-resetfactory-reset-run-system, run-hook (install-device), restart-system-to-run-mode

snapctl reboot --halt or --poweroff stores {"op": "halt"} or {"op": "poweroff"} on restart-system-to-run-mode. When that task completes, snapd halts or powers off instead of rebooting into run mode. Exactly one of the two flags is required.

Further reading

Roads from here

RUNNING SNAPS

  • Services — how snapd runs and tracks snap services
  • The overlord — changes and tasks and how they wait for each other
  • Refresh — refresh and the holds that remain

SNAPCTL · the snap's API into snapd

HOOKS · code snapd runs inside a snap

UBUNTU CORE · boot