Installing from source
make, make install for users for contributors
make joins image-garden's hand-written makefiles and the rules generated from its list of systems into image-garden.mk; make install puts the script, image-garden.mk, the release notes and the man page under a prefix, /usr/local by default, writing its directories and version into the script, which also runs uninstalled from the tree.
On the map: native
▶ Watch this deck — 9 scenes, ~6 min
Worth seeing first
- Snap, container or native package — how a source build compares with the snap, the container and packages
01 Building it yourself
make && sudo make install in a checkout of the tree. Nothing is compiled: the result is a shell script and image-garden.mk.
Maybe your distribution has no package, or its package is older than the release you need, or you're about to change image-garden yourself. Then build it from the tree. There's nothing to compile: image-garden is a shell script and one big makefile, generated as you build it. Two commands do it: make, then sudo make install. After that, image-garden make works as with any other install, with your system's QEMU. This deck shows what make builds, where install puts it, how to run the script straight from the tree, and the checks contributors run.
Details and sources
Details
- The README: "Follow your distribution's package installation method, or build from source:
make && sudo make install" (README.md:114-120). makebuildsimage-garden.mk(all:: image-garden.mk);make installinstalls it with the script (Makefile:20-50). Nothing in the tree is compiled;image-gardenis a/bin/shscript (image-garden:1).- The tools a source build needs at run time are in what the host needs.
- The README lists the tools a source build needs:
curl, the six emulatorsqemu-system-x86_64,-aarch64,-riscv64,-s390x,-ppc64and-loongarch64(or those of the architectures you use),qemu-imgandxorriso, and for the Ubuntu Core systems with a TPMswtpmand the Secure Boot firmware (README.md:122-126). - Every statement in this deck was checked by running
make,make installand the script on agit archiveof the pinned commit.
Read more
02 make builds one file for contributors in place of the general version
make runs rulegen.py once per data/*.ini, writes the systems list, and joins it, mk/*.mk and data/050-*.gen.mk into image-garden.mk.
make has one job here: build image-garden.mk. For each data file, such as ubuntu.ini, rulegen writes a makefile of generated rules, and a list of every system is written next to them. Then the hand-written makefiles from the mk directory and the generated ones are joined, in order, into one file. Its time is set to that of its newest input, not to the time of the build. You need Python for rulegen, and that's all; QEMU isn't needed yet. The result is the same file the snap and the container carry, and the one every image-garden make reads.
Details and sources
Details
data/050-%.gen.mk: rulegen.py data/%.iniruns./rulegen.py data/NAME.ini >data/050-NAME.gen.mk;data/000-systems.mkis written byrulegen.py --systems-metaover every.ini(Makefile:57-64).image-garden.mkiscat data/000-systems.mk, the sortedmk/*.mkand the sorteddata/050-*.gen.mk;touch --referencethen gives it the time of its newest input (Makefile:48-50).- Derived: rules in image-garden.mk compare a system's files with
$(MAKEFILE_LIST), so a time of the build would make every system's.urllook out of date after any rebuild; see how image-garden.mk is put together. rulegen.pyruns with Python 3; no QEMU tool is called while buildingimage-garden.mk(run for this map on a host with QEMU; derived for one without).image-garden.mk,data/*.gen.mkanddata/000-systems.mkare build output;make cleanremoves them (Makefile:23-27).
03 What make builds general version
make turns the tree's list of systems into rules and joins them with the hand-written ones into image-garden.mk. It needs Python, not QEMU.
make has one job here: build a single file, image-garden.mk. The tree carries a list of every system image-garden knows. make turns that list into rules, one set per system, and joins them, in order, with the rules written by hand, into that one file. Its time is set to that of its newest input, not to the time of the build. For this step you need make and Python, and that's all; QEMU isn't needed yet. The result is the same file the snap and the container carry, and the one every image-garden make reads.
Details and sources
Details
makewith no target buildsimage-garden.mk(Makefile:20-21). The list of systems is the tree'sdata/*.ini; the build writes one file of generated rules per data file and a list of every system's metadata, and concatenates that list, the hand-writtenmk/*.mkand the generated rules, each sorted, intoimage-garden.mk(Makefile:48-64).touch --referencegivesimage-garden.mkthe time of its newest input (Makefile:48-50). Derived: rules in image-garden.mk compare a system's files with$(MAKEFILE_LIST), so a time of the build would make every system's.urllook out of date after any rebuild.- The generator is a Python 3 program; no QEMU tool is called while building
image-garden.mk(run for this map on a host with QEMU; derived for one without). make cleanremovesimage-garden.mkand the intermediate files (Makefile:23-27).
Read more
- What the host needs, the container image, which runs the same build.
- Contributors: how image-garden.mk is put together, what rulegen writes.
04 What install puts where
prefix ?= /usr/local: bin/image-garden, include/image-garden.mk, share/doc/image-garden/, share/man/man1/image-garden.1. sed fills in @PREFIX@, @INCLUDEDIR@, @DOCDIR@ and @VERSION@.
sudo make install copies four kinds of file under a prefix, /usr/local unless you say otherwise. The image-garden script goes to its bin directory, image-garden.mk to include, the README and the release notes to the doc directory, and the man page to the manual. On its way in, the script is edited: the prefix, the include and doc directories and the version are written into it. That's how the command finds its makefile, in the include directory, and what image-garden version prints. Packagers add DESTDIR, to stage the files elsewhere, and usually a prefix of /usr.
Details and sources
Details
prefix ?= /usr/local,bindir,includedir = $(prefix)/include,datarootdir = $(prefix)/share,man1dir = $(datarootdir)/man/man1, andDESTDIR ?=(Makefile:3-14).installcopiesimage-gardento$(bindir)and runssed -i -e 's,@PREFIX@,$(prefix),' -e 's,@INCLUDEDIR@,$(includedir),' -e 's,@DOCDIR@,$(datarootdir)/doc/image-garden,' -e 's,@VERSION@,$(VERSION),'on it;image-garden.mkto$(includedir);README.md,NEWS,NEWS.snappy,NEWS.dockerand their.licensefiles to$(datarootdir)/doc/image-garden;image-garden.1to$(man1dir)(Makefile:29-46).- The script then has
GARDEN_PREFIX=PREFIX,GARDEN_INCLUDE=INCLUDEDIRandGARDEN_DOC=DATAROOTDIR/doc/image-garden, so anincludedirordatarootdirset apart from the prefix is followed (image-garden:10-12), andimage-garden makerunsmake -f "$GARDEN_INCLUDE"/image-garden.mk(image-garden:987-993). - Run for this map:
make install DESTDIR=… prefix=/usrstagedusr/bin/image-garden(withGARDEN_PREFIX=/usr),usr/include/image-garden.mk,usr/share/doc/image-garden/with seven files andusr/share/man/man1/image-garden.1, and the staged script printedimage-garden 0.8.0. Withincludedir=P/incthe script hadGARDEN_INCLUDE=P/incand found its makefile there (run at the pinned commit). - Because all three NEWS files are installed,
image-garden newsfrom a source install prints the main, the snap and the container notes (run).
Read more
05 Running it from the tree
Uninstalled, the script finds image-garden.mk and NEWS next to its real path. image-garden version prints image-garden @VERSION@.
You don't have to install it at all. Run the script from the checkout, after make, and it notices that nothing was written into it. It then looks for image-garden.mk and the release notes next to itself, following a symbolic link if you put one in your PATH. So image-garden make works straight from the tree, in any project directory. Only one thing gives it away: image-garden version prints the placeholder, @VERSION@, because the version is written in at install time. For a change of your own, that's the quickest loop: edit the tree, run make, try the system.
Details and sources
Details
- When
GARDEN_PREFIXis still the placeholder, the script setsGARDEN_ITSELF=$(readlink -f "$0")and uses its directory asGARDEN_PREFIX,GARDEN_INCLUDEandGARDEN_DOC: "This hacks supports both installed and uninstalled version equally" (image-garden:13-19). image-garden versionprintsimage-garden @VERSION@from an uninstalled tree;newsreadsNEWSnext to the script (image-garden:909-917). Both run for this map.image-garden make list-systems, run from another directory with the script of the tree, listed the systems; make warned about EDK2 firmware missing on that host, as any install does.makemust have builtimage-garden.mkfirst: the script runsmake -fon it (image-garden:987-993).~/binand~/projecton the slide are examples.
Read more
06 The project's own checks for contributors
make check: check-sh (shellcheck), check-py (mypy --strict), check-rulegen, check-dl-pkgproxy, check-allocate, check-rebase, check-dl-unpack-lock, check-pkgproxy-guest-shell.
Before you send a change, run make check. It runs shellcheck over the shell scripts, from image-garden itself to the snap's hooks and stubs, and mypy in strict mode over the Python. Then six tests: rulegen accepting good definitions and rejecting bad ones, image downloads through a stand-in pkgproxy, allocate against a fake SSH server, rebase onto a moved download, the lock around unpacking a shared download, and the pkgproxy setup that guests run. None of them boots a VM, though the rebase test needs qemu-img. Both checkers must be installed, or make stops and names the missing one. The container build runs the same make check.
Details and sources
Details
checkdepends oncheck-sh check-py check-rulegen check-dl-pkgproxy check-allocate check-rebase check-dl-unpack-lock check-pkgproxy-guest-shell(Makefile:66-67).check-shruns shellcheck overFILES_SH: the script,spread-chooser, the snap's hooks,qemu-stubs/,docker-stubs/dockerand the test scripts;check-pyrunsmypy --strictoverrulegen.pyand three test helpers (Makefile:69-97); the list also holdsqemu-stubs/virtiofsdandtests/ci/run-systems.sh. A missing checker is$(error program shellcheck is required)or… mypy is required(Makefile:90, Makefile:94).- The six tests:
tests/check-rulegen.sh(valid and invalid image definitions, and no Python traceback for a data mistake),check-dl-pkgproxy.sh(downloads through a stand-in pkgproxy,tests/dl-pkgproxy-server.py),check-allocate.sh(ports and SSH, withtests/fake-sshd.py),check-rebase.sh(rebase of a prepared image whose download moved to anotherGARDEN_DL_DIR, named with and without.qcow2; it runsqemu-img, tests/check-rebase.sh:1-39),check-dl-unpack-lock.sh(concurrent unpacking of one download, "No network, no KVM"),check-pkgproxy-guest-shell.sh(the guests' pkgproxy setup under dash and busybox sh) (Makefile:99-115). - The container's
build-gardenstage installsqemu-utilsamong its tools and runsmake checkbeforemake install(Dockerfile:58-71). make fmtformats with shfmt and black (Makefile:117-126).
Read more
07 When the makefile is missing
make: INCLUDEDIR/image-garden.mk: No such file or directory means the script cannot find its makefile where make install put it. Install both files together, with the same prefix and includedir.
One mistake gives a message that's easy to misread. The installed script looks for image-garden.mk in one place only: the include directory that make install wrote into it. That's the prefix's include directory, or the includedir you gave. Copy the script alone to another machine, or remove the makefile, and image-garden make fails before doing anything: make says the makefile at that path doesn't exist, and that there's no rule to make it. Your project isn't missing a file; image-garden is. Run make install again on that machine, so that the script and its makefile arrive together.
Details and sources
Details
- The installed script looks in
@INCLUDEDIR@, whichmake installreplaces with$(includedir), the directory it copiesimage-garden.mkto;@DOCDIR@becomes$(datarootdir)/doc/image-garden, where the release notes go (image-garden:10-12, Makefile:12, Makefile:29-40). An uninstalled script looks next to itself instead (image-garden:13-19). - Run for this map: after
make install prefix=P includedir=P/incthe script hadGARDEN_INCLUDE=P/inc, andimage-garden make list-systemslisted the systems. WithP/inc/image-garden.mkremoved, andLC_ALL=C, it printedmake: P/inc/image-garden.mk: No such file or directoryandmake: *** No rule to make target 'P/inc/image-garden.mk'. Stop.The slide uses/opt/igforP. - A script copied without its makefile behaves the same (derived).
image-garden makerunsmake -f "$GARDEN_INCLUDE"/image-garden.mk(image-garden:987-993).
Read more
08 Where to go from here general version
Next: what the host needs, distribution packages, the container image and image-garden make.
So a source build is two commands: make builds one file from the tree, and install puts it and the script under a prefix, with the version written in. Next, the host tools deck lists what you need to install alongside: the emulators, curl, xorriso and the rest. The packages deck covers builds others make for you, and the container deck one more build of the same tree. And the command line deck shows what to run once it's installed.
Details and sources
09 Where to go from here for contributors in place of the general version
Next: what the host needs, how image-garden.mk is assembled, rulegen, distribution packages, the container image and image-garden make.
So a source build is two commands: make joins the rules into one file, and install puts it and the script under a prefix, with the version written in. Next, the host tools deck lists what you need to install alongside. The assembly deck shows how image-garden.mk is put together, file by file, and the rulegen deck what it writes for a system. The packages deck covers builds others make for you, the container deck one more build of the same tree, and the command line deck what to run next.
Details and sources
Read more
Basis of the statements on this page
Read in the Makefile, image-garden and README of image-garden at the pinned commit ("code"), and checked by running make, make install (with DESTDIR, prefix and includedir) and the uninstalled script on a git archive of that commit ("run").
Targets
| Target | What it does |
|---|---|
make (all) | builds image-garden.mk from data/*.ini and mk/*.mk |
make install | installs the script, image-garden.mk, the docs and the man page |
make clean | removes image-garden.mk, data/050-*.gen.mk and data/000-systems.mk |
make check | shellcheck, mypy --strict and six tests (one needs qemu-img) |
make fmt | shfmt and black over the scripts |
make dist | image-garden-VERSION.tar.gz of the sources |
make docker-test-smoke | builds the container image and runs three smoke tests |
Install locations
| Variable | Default | Holds |
|---|---|---|
prefix | /usr/local | everything below |
bindir | $(prefix)/bin | image-garden |
includedir | $(prefix)/include | image-garden.mk; written into the script, which looks there |
datarootdir | $(prefix)/share | doc/image-garden/: README.md, NEWS, NEWS.snappy, NEWS.docker and their licences; written into the script for image-garden news |
man1dir | $(datarootdir)/man/man1 | image-garden.1 |
DESTDIR | empty | a staging root, for packaging |
Examples
make && sudo make install # /usr/local make install DESTDIR=$PWD/stage prefix=/usr # stage a package's files ./image-garden version # uninstalled: image-garden @VERSION@ make check # needs shellcheck and mypy
Messages
| Message | Cause |
|---|---|
program shellcheck is required | make check without shellcheck (or mypy) |
make: INCLUDEDIR/image-garden.mk: No such file or directory | the script's makefile is not where make install put it |
image-garden @VERSION@ | the script runs uninstalled, from the tree |
Roads from here
RUNTIME · make rules and the CLI
- image-garden make and the other commands — the commands you run once it is installed
CONTAINER · OCI image
- The OCI container — the container, which builds and checks the tree the same way
NATIVE · packages
- What a native install needs from the host — the tools and emulators a source build needs on the host
- Distribution packages — the distribution packages built from the same tree
BUILD TIME · data and rules
- How image-garden.mk is put together — how image-garden.mk is put together, file by file
- What rulegen.py emits — what rulegen.py writes for each system