Working on the serial console

why the TTY acts oddly for users

A .run script starts QEMU with -nographic, so the guest's serial console and the QEMU monitor share your terminal. The guest does not learn your terminal size, Ctrl-C goes to the guest, Ctrl-A starts QEMU's own keys, and allocate with SSH is more comfortable for long work.

On the map: runtime

▶ Watch this deck — 7 scenes, ~5 min

Worth seeing first

Part of journeys

01 No window, just a serial line

THE SERIAL CONSOLE your terminal qemu … -nographic the guest's console its first serial port like a serial cable plugged into a server in a rack works anywhere, even over SSH to a build machine without a screen simpler than the terminal you know: a few things behave differently
Your terminal is connected through QEMU started with -nographic to the guest's first serial port, which is the guest console; like a serial cable into a server, it works even over SSH to a build machine without a screen, but a few things behave differently

The default display option is -nographic: the guest's serial console is your terminal.

When you run a .run script, no window opens. The script starts QEMU with -nographic, so the machine's first serial port is wired to your terminal, and the guest uses that port as its console. It's like plugging a serial cable into a server in a rack. That works anywhere, even in an SSH session to a build machine with no screen at all. But a serial line is a simpler thing than the terminal you're used to, and a few things behave differently. This deck shows what to expect, and how to get around it.

Details and sources

Details

  • QEMU_DEFAULT_DISPLAY_OPTION ?= -nographic (010-qemu.mk:13); the script uses it unless QEMU_DISPLAY_OPTION is set (010-qemu-x86_64.mk:66, 010-qemu-x86_64.mk:99).
  • With -nographic, QEMU has no graphical output and connects the first serial port to standard input and output. Derived: this is how QEMU's -nographic mode and a serial line behave in general; image-garden's source sets the option but does not describe the behaviour.
  • That the guest's console is on its first serial port is a property of the cloud images, not set by image-garden (derived).

Read more

02 Two things on one terminal

TWO LISTENERS your keys the guest: most keys QEMU: after Ctrl-A Ctrl-A H list QEMU's keys Ctrl-A C to the QEMU monitor, and back Ctrl-A X quit QEMU on the spot Ctrl-A Ctrl-A one Ctrl-A for the guest
Most keys go to the guest; keys after Ctrl-A go to QEMU: Ctrl-A H lists the keys, Ctrl-A C switches to the QEMU monitor and back, Ctrl-A X quits QEMU, Ctrl-A Ctrl-A sends one Ctrl-A to the guest

Ctrl-A H help, Ctrl-A C monitor and back, Ctrl-A X quit QEMU, Ctrl-A Ctrl-A sends one Ctrl-A to the guest.

Your terminal is shared by two listeners. Most keys go to the guest. But QEMU keeps one key for itself, Control A, and whatever you press right after it is a command for QEMU, not for the guest. Control A, then H, prints the list. Control A, then C, switches to the QEMU monitor, where you can ask QEMU about the machine, and the same keys switch back. Control A, then X, quits QEMU on the spot. And if you need to send a real Control A to the guest, say for the shell, press it twice.

Details and sources

Details

  • Derived: this is how QEMU's -nographic mode and a serial line behave in general; image-garden's source sets the option but does not describe the behaviour. Under -nographic, QEMU multiplexes the serial port and its monitor on the terminal; Ctrl-A is the escape key, Ctrl-A H prints the keys, Ctrl-A C switches between the console and the monitor, Ctrl-A X exits QEMU, Ctrl-A Ctrl-A sends one Ctrl-A.
  • The default comes from 010-qemu.mk:13.
  • Ctrl-A is also the shell's "start of line" key and screen's or tmux's prefix in many setups; inside the guest, press it twice.
  • allocate turns the monitor off with -monitor none and sends the serial port to a file (image-garden:1155-1157).

Read more

03 Control C goes to the guest

CONTROL C Ctrl-C QEMU keeps running the guest's program is interrupted $ sudo shutdown -h now a clean end Ctrl-A X pulls the power cord without -snapshot, half-written changes stay in the image
Ctrl-C travels down the serial line to the program running in the guest and QEMU keeps running; sudo shutdown -h now ends the machine cleanly; Ctrl-A X stops QEMU at once like pulling the power cord, and without -snapshot half-written changes stay in the image

Ctrl-C reaches the guest, not QEMU. Shut down from inside; Ctrl-A X stops QEMU at once, like pulling the plug.

Here's the one that surprises people. You press Control C to stop QEMU, and nothing stops. That's because Control C goes down the serial line like any other key, to whatever is running in the guest. To end the machine, shut it down from inside, with sudo shutdown -h now. Control A, then X, also works, but it's like pulling the power cord: the guest gets no chance to finish writing. Without -snapshot, those half-written changes stay in the image.

Details and sources

Details

  • Derived: this is how QEMU's -nographic mode and a serial line behave in general; image-garden's source sets the option but does not describe the behaviour. In -nographic mode the terminal is raw, so Ctrl-C is sent to the guest as a character.
  • Shutting down from inside: "From inside VM: sudo shutdown -h now" (README.md:81-82).
  • Ctrl-A X ends QEMU at once; without -snapshot, writes the guest had not flushed are lost and the image keeps whatever reached it (README.md:85-86 for the persistent default; the rest derived).

04 The guest does not know your terminal size

TERMINAL SIZE your terminal characters 160 columns × 50 rows resized: the guest never hears the guest guesses 80 columns × 24 rows long lines wrap early; editors and top draw in a corner host$ stty size 50 160 guest$ stty rows 50 cols 160 guest$ resize where the command is installed
The serial line carries only characters, so when your terminal is 160 by 50 the guest still guesses 80 by 24; lines wrap early and full-screen programs draw in a corner; run stty size on the host, then stty rows and cols in the guest, or resize where installed

Run stty size on the host, then stty rows R cols C in the guest, or resize where it is installed.

A serial line carries characters, and nothing else. When you resize your terminal window, the guest never hears about it, so it guesses, usually eighty columns by twenty-four rows. Long command lines wrap in the wrong place, and full-screen programs like an editor or top draw in one corner, or scroll in odd ways. The fix is to tell the guest yourself. Run stty size on your own machine to see the numbers, then stty rows and cols with those numbers inside the guest. If the guest has the resize command, it can work them out for you.

Details and sources

Details

  • Derived: this is how QEMU's -nographic mode and a serial line behave in general; image-garden's source sets the option but does not describe the behaviour. A serial line has no way to send the window size, so the guest keeps its default, commonly 80 by 24, until told otherwise.
  • stty size prints rows then columns; stty rows R cols C sets them for the guest's terminal. resize comes with xterm and may not be installed in a cloud image.
  • The numbers on the slide are an example.
  • The default display option that leads here is 010-qemu.mk:13.

05 Messages land in your typing

MESSAGES IN YOUR TYPING ubuntu@ubuntu-24-04:~$ sudo apt ins [ … a kernel message … ] tall hello the command is still there, split up Ctrl-U clear it Enter new prompt ubuntu-cloud-24.04.x86_64.qcow2.log the provisioning boot, as make saw it ubuntu-cloud-24.04.x86_64.PORT.serial.log a machine started by allocate
A kernel message lands in the middle of a command being typed on the console; Ctrl-U clears the line and Enter gives a fresh prompt; the provisioning boot's console is saved in X.qcow2.log and an allocated machine's in X.PORT.serial.log

Kernel messages share the console. The provisioning boot's console is in X.qcow2.log; an allocated machine's in X.PORT.serial.log.

The console is also where the kernel and the boot write their messages. They don't wait for you to finish a line, so a message can land right in the middle of a command you're typing. Your command is still there; press Control U to clear it, or Enter to get a fresh prompt. This is the same console make watched while it prepared the image, and it saved all of it in the qcow2.log file next to the image. Machines that allocate starts write it to a serial log of their own, named with the port.

Details and sources

Details

  • Derived: this is how QEMU's -nographic mode and a serial line behave in general; image-garden's source sets the option but does not describe the behaviour. The kernel writes its messages to the console it was given, whatever is being typed there.
  • The provisioning boot's output is piped through tee X.qcow2.log (rulegen.py:604-609).
  • allocate starts the script with -serial file:X.PORT.serial.log (image-garden:1153-1163), so an allocated machine's console goes to that file.

06 When you want something more comfortable

SOMETHING MORE COMFORTABLE the serial console fine for a quick look image-garden allocate + ssh knows your terminal size follows it when you resize as many sessions as you like copies files with scp Ctrl-C behaves as usual QEMU_DISPLAY_OPTION="-display gtk" a window
The serial console is fine for a quick look; for real work image-garden allocate and SSH give a terminal that knows and follows its size, many sessions, scp and an ordinary Ctrl-C; QEMU_DISPLAY_OPTION=-display gtk opens a window instead

For long work use image-garden allocate and SSH. For a window, QEMU_DISPLAY_OPTION="-display gtk".

For a quick look around, the serial console is fine. For real work, start the machine with image-garden allocate and log in over SSH instead. An SSH session knows your terminal size, follows it when you resize the window, lets you open as many sessions as you like, and copies files with scp. Control C behaves as usual there, too. And if you'd rather see a screen, set QEMU_DISPLAY_OPTION to -display gtk, and QEMU opens a window with the machine's display.

Details and sources

Details

  • allocate sets QEMU_DISPLAY_OPTION to -display none unless you set it, and forwards a local port to the guest's SSH port (image-garden:1105-1107).
  • "Note that you cannot use ssh to connect to a machine that was started with the run script" (README.md:86-89).
  • The README suggests -display gtk for a graphical display (README.md:928-929).
  • SSH's handling of the window size, sessions and scp is ordinary OpenSSH behaviour (derived).

07 Where to go from here

ROADS The serial console image-garden allocate in the background, over SSH The .run script the display option and more Overlay and first boot the console, captured
Three roads from Working on the serial console: image-garden allocate, the .run script and its variables, and the overlay and provisioning boot

Next: image-garden allocate, the .run script's variables, and the provisioning boot.

So the console is a serial line: QEMU's keys start with Control A, Control C goes to the guest, the guest guesses your terminal size until you tell it, and boot messages share the screen with you. When that gets in the way, take allocate and SSH. Back in the .run script deck, you'll find the display option and the other variables. And the overlay deck shows the same console, captured while make prepared the image.

Details and sources

Details

  • Everything on this deck applies to .run scripts started by hand; allocate starts them with no display, no monitor and the console in a file.

Read more

Basis of the statements on this page

The default display option was read in the image-garden source at the commit in garden.commit ("code"). The keys and terminal behaviour are QEMU's and a serial line's general behaviour ("derived"); image-garden does not change them.

Console survival

WherePress or typeWhat happens
consoleCtrl-A HQEMU lists its keys
consoleCtrl-A Cswitch to the QEMU monitor; again to switch back
consoleCtrl-A XQEMU exits at once, like pulling the plug
consoleCtrl-A Ctrl-Aone Ctrl-A reaches the guest
consoleCtrl-Cinterrupts the program in the guest, not QEMU
consoleCtrl-U, Enterclear a line broken up by a message, or get a fresh prompt
hoststty sizeprints your terminal's rows and columns
gueststty rows R cols Ctells the guest the size
guestresizeworks out the size itself, where installed
guestsudo shutdown -h nowa clean end: the guest powers off and QEMU exits
hostQEMU_DISPLAY_OPTION="-display gtk" ./X.runa window instead of the serial console
hostimage-garden allocate X then ssh -p PORT user@localhosta normal terminal over SSH

Where the console goes

How the machine was startedConsole
./X.runyour terminal (-nographic, 010-qemu.mk:13)
provisioning boot by makeyour terminal and X.qcow2.log (rulegen.py:604-609)
image-garden allocateX.PORT.serial.log, no monitor (image-garden:1155-1157)

Roads from here

RUNTIME · make rules and the CLI