Open Firmware Linux Debug Tour

Linux and the Open Firmware PC Demo

This document tells you how to run Linux from the PC demo version of FirmWorks Open Firmware. A lot of the information herein also applies to using Linux with any Open Firmware version from FirmWorks.

Getting the Demo

Use of the demo is subject to the terms of a no-charge Evaluation License. Do not download demo unless you agree to abide by the terms of that license.

Download demo.zip (approximate size 2.5 Mbytes) to get the demo.

Making the Demo Floppies

Two of the files in the demo archive file, floppyof.img and floppylx.img, are verbatim floppy disk images. Each of those files must be copied verbatim onto a blank floppy. The image files contain the floppy filesystem format within them, so the copy must be done in a block-for-block fashion. If you are doing it on a Linux system, the appropriate command for the copy is dd.

Get two blank floppy disks
Insert the first one into the floppy drive
dd if=floppyof.img of=/dev/fd0 bs=36b
After the operation has finished and the drive light has gone out, remove and label the first floppy "Open Firmware"
Insert the second floppy
dd if=floppylx.img of=/dev/fd0 bs=36b
After the operation has finished and the drive light has gone out, remove and label the second floppy "Linux"

The Open Firmware floppy (floppyof.img) contains

A special boot sector that uses PC BIOS calls to read the Open Firmware image into memory.
An MS-DOS file system with ...
The file OFW.IMG , which is the Open Firmware image
The small file NVRAM.DAT, which contains Open Firmware configuration information that you can change

You can put the Open Firmware floppy into a stock PC that is configured (from the BIOS setup screen) to boot from floppy, and the PC will run Open Firmware instead of the regular operating system.

The Linux floppy (floppylx.img) contains

An MS-DOS file system with ...
The file \VMLINUZ , which is a gzip'ed ELF file containing the Linux kernel
The file \INITRD.IMZ, which is a gzip'ed RAMdisk image of the Linux root filesystem

This Linux floppy cannot be booted directly from the BIOS, because it lacks the special helper programs (boot loader, etc) that are normally present on a Linux boot floppy. Instead, it contains just the Linux kernel, which Open Firmware can load directly, without all the helper programs. For more information, see Which Linux File? .

The initfd.imz RAMdisk image is from the Linux Router Project. The kernel is built from the 2.2.16 sources with a patch to preserve the Open Firmware debugger function (see Using the Debugger ) and another patch that is similar in spirit to the initrd-always patch from the Linux Router Project.

Running the Programs

Use your PC's BIOS setup facility to configure it to boot from floppy disk.
Insert the Open Firmware floppy
Turn on the PC
After the BIOS reads the floppy, you should see a series of startup messages on the screen*, then an "ok" prompt
Insert the Linux floppy
Type linux
You should see some loading messages, then a splash screen, then an "ok" prompt
Type go to run Linux, or use the debugger

* Normally, production versions of Open Firmware are configured to boot quietly, often in two seconds or less, but the demo version has all the startup reports enabled for ease of debugging. See Open Firmware Startup Messages .

The linux command is a variant of the standard Open Firmware load command; it loads a ramdisk image in addition to the operating system kernel. If you wanted to load just the kernel, you could use load or boot , which is equivalent to load then go .

Setting Configuration Variables

Open Firmware has several configuration variables that control the Linux booting process. (Normally OFW configuration variables are stored in non-volatile RAM or in a writeable sector of FLASH memory, but on systems where OFW doesn't have access to such memory they can be stored in a disk file. On the floppy demo version, they are stored in NVRAM.DAT on the floppy.)

The OFW boot command (and its variants load and linux ) can specify the device and file from which to load the Linux kernel, as well as the arguments to pass to the kernel as its "cmdline" string. The complete form of the boot command is:

boot device-specifier arguments...

device-specifier indicates the device and file from which to load the kernel - for example disk:\boot\vmlinuz or nfs:\\myserver\kernels\vmlinuz . The remainder of the line ( arguments... ) is passed to the kernel as the cmdline - for example console=ttyS0,9600 root=/dev/ram0 .

If either device-specifier or arguments... (or both) is missing, the boot command substitutes configuration variable values for the missing information. The value of the boot-device configuration value supplies the default for device-specifier and the value of boot-file for arguments... . (It is possible to omit device-specifier but still supply arguments... on the command line. If the first word on the command line after boot or load does not have the form of a device pathname and is not a known alias, it is treated as part of arguments... ) (The name boot-file is an historical artifact from before the time Open Firmware supported filesystems, when it had to depend on intermediate programs for that function. The first command line argument to such intermediate filesystem reader programs was typically a file name.)

boot-device and boot-file are generic Open Firmware features whose usage is not specific to Linux. There is an additional Linux-specific configuration variable named ramdisk . If its value is set to a nonempty string, the linux command, whose arguments are the same as for boot and load , will first load, and uncompress if necessary, an initial RAMdisk image from the device and file specified by the value of ramdisk .

You can inspect the value of a configuration variable with, for example:

printenv varname

You can change its value with:

setenv varname value
   or
editenv varname

editenv is usually the most convenient - it puts you into an interactive editor so you can modify the existing value. editenv will also let you set the value to the empty string, which setenv can't do.

For the floppy-loaded PC demo version, the default value of boot-device is "a:\vmlinuz" and the default value of ramdisk is "a:\initrd.imz". "a" is a device alias that expands to the first floppy device. To load Linux from a different place, you could set boot-device as in these examples.:

setenv boot-device c:\boot\vmlinuz
setenv boot-device cdrom:\vmlinuz
setenv boot-device /pci/scsi/disk@4:3,\boot\vmlinux
setenv boot-device net
setenv boot-device nfs:\\myserver\mypath\vmlinux.gz
setenv boot-device http:\\168.192.0.5\binaries\vmlinux

Similarly, you could change ramdisk to load the initrd image from a different place. If you don't want to load the initrd image at all, just use the load command instead of the linux command.

The various pathnames are shown to illustrate the range of booting choices that Open Firmware can provide. Some of these devices aren't available in the demo version. In particular, the network booting options (nfs, http) require a network card driver, which the PC demo version does not include by default.

Using the Debugger

OFW's assembly-language debugger can step through and breakpoint the Linux startup sequence. However, if you are building your own kernel, you must apply a small patch so that it does not disable the debugger (the kernel on the demo floppy already has the patch). The patch files are in the demo archive file. The patch for 2.2.x kernels (patch-2.2.txt) is almost identical to the patch for 2.4.x kernels (patch-2.4.txt); they differ from one another only in small details of the surrounding context. The patch avoids zeroing the %eflags register (which would disable the debugger) and leaves the Open Firmware handlers for the debug and breakpoint interrupts installed in the interrupt table. Open Firmware can boot Linux without the patches; the patches are only necessary if you wish to use the OFW debugger on the Linux kernel.

If the Linux kernel file contains symbols (i.e. it has not been stripped) and OFW's client-symbols? configuration variable is set to true (its default value), then OFW will load the symbols into its memory area when it loads the kernel. The disassembler will then use symbolic names when displaying addresses and you can use symbolic names for addresses that you enter via the command interpreter.

Here is an example of debugging Linux with Open Firmware.

ok linux
Loading ramdisk image from disk:\initrd.imz ...
Boot device: /isa/fdc/disk@1:\vmlinuz
Arguments: root=/dev/ram0 mount_ramdisk=1

Linux start address at c0100000
Type 'go' to start or 'help-debug' to see debugger commands.
ok %eip dis
c0100000 _text        cld
c0100001 _text+1      mov     eax,18
c0100006 _text+6      mov     es,eax
c0100008 _text+8      mov     es,eax
(etc.)
ok step
c0100001 _text+1      mov     eax,18
ok .registers
EIP: c0100001   Flags: 6 vrn0oditszaPc

        EAX  ECX  EDX   EBX      ESP   EBP      ESI  EDI
   ffc345d8    0    0     0 ffcca770     0    90000    0

  ES: 18   CS: 10   SS: 18    DS: 18    FS: 18   GS: 18

ok start_kernel dis
c01ecb50 start_kernel        push    edi
c01ecb51 start_kernel+1      push    ebx
c01ecb52 start_kernel+2      sub     esp,10
(etc.)
ok start_kernel till
c01ecb50 start_kernel        push    edi
ok kbd_init till
<Various console messages from Linux>
c01f3ad4 kbd_init            and     eax,-7d000000
ok %eip dis
c01f3ad4 kbd_init            and     eax,-7d000000
c01f3ad9 kbd_init+5          or      eax,50000000
...
c01f3b39 kbd_init+65         add     esp,c
c01f3b3c kbd_init+68         ret     near
ok kbd_init 68 +  to %eip
ok step
c01f3da5 tty_init+1bd        call    c01f4704 rs_init
ok hop
c01f3daa tty_init+1c2        call    c01f426c pty_init
ok ^R
^R
^R
go

Use the command that loads the ramdisk too






Disassemble starting at %eip





Single step

Display registers
  In the Flags display,
  lower case means the bit
  is clear, upper case means
  the bit is set


Disassemble from start_kernel
  Typing a symbol name
  pushes its value on the
  Forth stack.

Execute until start_kernel

Execute until kbd_init


Examine the kbd_init routine, 
look for its end,



and set %eip to that address,
thus skipping the routine,
otherwise Linux takes over the keyboard


Hop goes over instead of into subroutines

Control-R toggles the screen buffer so you
can see both Linux output and Open Firmware
output after Linux has scrolled the screen

In this example, after you continue with go, Linux will continue with its startup sequence. If everything is okay, it will come up to the login prompt, but you won't be able to login because in this example we forced Linux to skip its keyboard initialization step so the firmware debugger could continue to use the keyboard. An alternative would be to use a serial line for either the firmware console or the Linux console. If you were using a serial line for the firmware console and you needed to debug past the point where Linux normally takes over the serial port hardware, you would need to make Linux skip that step, thus preserving the state of that port for the firmware debugger.

Which Linux File?

The Linux build process creates several different images. At the top level of the Linux source/build tree (e.g. /usr/src/linux/., not the root of the overall filesystem) there is a file named vmlinux that is in ELF format. That file is the Linux kernel itself. Deeper down the tree, in arch/i386/boot/zImage (or bzImage) and arch/i386/boot/compressed/vmlinux , there are other files that files contain a stripped, compressed copy of the Linux kernel code prefixed by other little programs that do things like getting the CPU into protected mode and uncompressing the kernel. Normally, when you boot Linux from a floppy, the floppy must have the zImage or bzImage file on it.

However, that is not the case for Open Firmware. All of those extra little programs are unnecessary because:

Open Firmware already runs in 32-bit protected mode, so there's no need to switch to that mode
Open Firmware can uncompress gzip'ed images
Open Firmware understands the ELF format and can load the various sections directly to the locations the ELF header specifies
Open Firmware understands and uses virtual addressing, so it can pre-establish the correct associations between virtual and physical addresses as needed by the Linux kernel (in particular, it can load Linux at physical address 0x100000 and create an additional mapping so virtual address 0xC0100000 refers to that physical address)
Open Firmware can create and populate the bootup parameter block through which the Linux kernel receives system information like the memory configuration and the command line options
Open Firmware understands ELF symbol tables, so if it loads a kernel with symbols, they are available to the OFW debugger
Open Firmware understands several different file system formats, including ext2, MSDOS, and ISO9660, and network protocols like NFS, DHCP, TFTP, and HTTP, so it can load Linux directly from a file on most common media types.
Open Firmware has an interactive command interpreter and a facility for managing and remembering command line options, so programs like LILO are unnecessary.

Open Firmware cannot execute a zImage or bzImage file, because the little "helper" programs at the beginning of those files assume that they have been loaded directly by BIOS and that they begin execution in a real-mode environment.

Instead, Open Firmware directly boots the ELF-format vmlinux file, whether gzip'ed or not, whether or not symbols have been stripped. When Open Firmware loads a gzip'ed file, it automatically uncompresses it before preparing it for execution as specified by the ELF header.

Therefore, when you build a new Linux kernel, the file you need for Open Firmware is the vmlinux file at the top directory of your build tree. The final make command can be just "make vmlinux"; you don't need "make zImage" or "make bzImage" (it's okay to make zImage, because that makes vmlinux too, but the additional steps involved in the creation of zImage are unnecessary). Once you have the vmlinux file, you just copy it to some medium that Open Firmware can read. If you wish, you can gzip it first. It doesn't matter to OFW what you name the file, because OFW determines the file format from its contents, not by its name. Nevertheless, it's probably a good idea to name a gzip'ed version of the kernel something like vmlinux.gz or vmlinuz, as a clue to humans.

Here is a recipe for putting a Linux image on a floppy disk for Open Firmware:

Start with a blank floppy disk formatted with an MS-DOS filesystem (or format it with e.g. mkdosfs)
cd to the top directory of the Linux source tree
Perform any necessary configuration steps if you have not already done so ("make config", "make depend", etc.)
Then,

make vmlinux
gzip -c vmlinux >vmlinuz
mcopy -o vmlinuz a:

You can put an initial ramdisk image (initrd) on the same floppy disk if you have room. It will probably be necessary to compress it because uncompressed ramdisk images are usually too large for a floppy. Assuming that you have already created the initrd.img file,

gzip -c initrd.img >initrd.imz
mcopy -o initrd.imz a:

Open Firmware Startup Messages

Summary

These messages that are displayed during the startup of this BIOS-loaded demo version of Open Firmware are as follows:

The PC BIOS loads and executes the boot sector from the floppy. That boot sector displays the character "a" then proceeds to load the OFW.IMG file from the floppy using BIOS interrupts
The boot sector program displays a "." character as it loads each block of the OFW.IMG file. If the PC BIOS has a good floppy driver, the loading will proceed quickly. If not, the loading may take a long time, just a few blocks per second.
When OFW.IMG is in memory, the boot sector program jumps to its entry point.
The first thing the OFW image does is switch from 16-bit real mode to 32-bit protected mode. The assembly language code that does so displays numbers from 1 to 7 in the upper left hand corner of the screen. Typically this happens so fast that you can't see it happening. If something goes wrong and the system hangs during this sequence, you will see the last number that it got to before the hang. Otherwise you'll probably just see the number 7.
Once in 32-bit protected mode, the Open Firmware begins to execute the inititialization chains, which are written in Forth.Various initialization steps report their progress with messages that are displayed in violet text on a blue background. Those messages are also sent to the COM1 port at 9600 baud.
Upon successful startup, Open Firmware switches the screen to a white background, displays a banner, and issues the "ok" prompt.

Details

Here is a blow-by-blow of the startup reports on the screen:

     a...........................................

The above comes from the boot sector code that the BIOS loads from the first two sectors of the floppy. That code is a tiny little program that knows how to read OFW.IMG from the DOS file system on the floppy. (It is also supposed to know how to read OFW.IMG from the hard driver if there is a file named C:\OFW.IMG, but that doesn't work with the new-style FAT32 filesystem that is uses on modern large disks. It might work if your C: partition is in FAT16 format.)

The 'a' comes out right when the boot sector starts, then a dot is displayed every time the code reads a sector from the OFW.IMG file. Since the OFW.IMG file is about 300K, there will be several lines of dots. On some PCs, the dots just fly across the screen and the floppy loading process is over in a few seconds. On others, the dots come out a few per second, and the process is painful. I think that some BIOS floppy drivers tend to blow revs.

After several lines of dots, the OFW.IMG file is in memory (at address 0x7c00) and the boot sector code jumps to it. The CPU is still in real mode at this point, executing 16-bit code.

The next section of code, which is located at the beginning of the OFW.IMG file, is responsible for determining how much memory there is (via the use of real-mode BIOS calls) and then getting the processor into 32-bit protected mode.

The progress reports from this section of code are displayed in the upper left-hand corner of the video screen. They consist of the numbers 1-7, displayed in a funny color. The are all displayed at the same position, and if all goes well it won't take long for it to get all the way up to 7, so all you are likely to see is the number 7 up there in a funny color. Here is what happens as the numbers are being displayed:

Display Actions

Turn off floppy motor

Determine memory size with BIOS calls

1

Turn off interrupts

2

Disable NMI

3

Init COM1 at 9600,8,n,1

Enable A20 using port 92

4

Enable A20 using the keyboard controller

5

Output CR, LF, 'F' on COM1

Switch to protected mode, which is a bit complicated

6

Output 'o' 'r' on COM1

Copy the rest of OFW.IMG from low memory to 0x20.0000 (2 MB)

7

Jump to the copy

Display	Actions
	Turn off floppy motor
	Determine memory size with BIOS calls
1
	Turn off interrupts
2
	Disable NMI
3
	Init COM1 at 9600,8,n,1
	Enable A20 using port 92
4
	Enable A20 using the keyboard controller
5
	Output CR, LF, 'F' on COM1
	Switch to protected mode, which is a bit complicated
6
	Output 'o' 'r' on COM1
	Copy the rest of OFW.IMG from low memory to 0x20.0000 (2 MB)
7
	Jump to the copy

After this, everything is 32-bit code that is in the Forth dictionary. The first code to execute is a little machine language sequence ( prom-cold-code ) that sets up the Forth stack pointers and user area and then starts running Forth code. The top-level Forth entry point is the word "cold", which in turn runs the usual set of initialization chains, the same as for all of our implementations.

Defer Word Implementation chain

init-io stand-init-io

do-init init

init-environment stand-init

cold-hook startup

Defer Word	Implementation chain
init-io	stand-init-io
do-init	init
init-environment	stand-init
cold-hook	startup

As usual, stand-init-io initializes the memory allocator and turns on the diagnostic output device. The diag output device used to be COM1, but today I modified it so that early startup messages go both to COM1 and also to the video screen, using a funny color.

The first diagnostic output is

    Type 'i' to interrupt stand-init sequence

which happens as the very last thing in the "init" chain. Said 'i' has to come from COM1, because the PC keyboard driver can't be installed this early, and we can't call the BIOS to get keyboard input because we are running in 32-bit protected mode.

Most of the rest of the startup messages come from the stand-init chain, until you get the "probe-pci" message, which is the first message from the "startup" word. Shortly after probe-pci , startup does "install-console", at which point the screen changes color to black-on-white.

Then you get the banner and an "ok" prompt.