“Crystal sticks the boot in to her UEFI-based workstation”
Preparing bootable optical media for UEFI-based X86 machines
Booting an X86 system from a CD, DVD, or blu-ray disc has been possible for many years.
However, many modern X86 systems no longer support the traditional BIOS interface and instead offer an exclusively UEFI-based boot environment.
In this article, Crystal explains how to master optical discs with EFI boot code on an OpenBSD system.
Preamble
I've previously written about using blu-ray discs for data storage, but the focus in that article was on streaming an encrypted tar archive to and from the optical media, mostly for backup and archiving purposes.
Another use for recordable optical media is creating a bootable disc with the installation files for the OpenBSD system. Although USB flash drives can also be used for this purpose, write-once media has the advantages of durability and not being vulnerable to accidental or deliberate over-writing. In the case of BD-R, it can also be surprisingly fast.
Of course, pre-built disc images suitable for writing to a CD-R are available from the OpenBSD project. However, with literally gigabytes of additional space on a BD-R disc, we can include extra files of our own choice, such as the source code, any available errata patches, documentation, binaries for other architectures, multiple OpenBSD versions, distfiles for any specific ports that we are interested in building, or even pre-built binary packages from the ports tree.
This obviously requires creating a custom layout of the required files and writing it to suitable media, with the added complication of making the disc bootable.
A historical perspective
If you've created bootable optical discs on an OpenBSD machine in the distant past, you might be familiar with the -b flag to mkhybrid:
The -no-emul-boot flag is required here, as unlike mkhybrid, mkisofs doesn't make this assumption based on the size of the file specified with -b.
Fun fact!
Default boot catalog location:
We don't need to specify the -c option to either program, as despite what the manual pages say, without it the boot catalog will be written to /boot.catalog, (or /BOOT.CAT when reading the disc without rock ridge extensions).
The commands above generate a disc image which is essentially just a regular ISO-9660 filesystem, with rock ridge extensions for long filenames and other file attributes typical of a BSD system, and also with boot code based on the El Torito specification.
However, this simple approach fails to create a disc which will be bootable on modern X86 systems which no longer support the traditional BIOS interfaces for booting from optical media and instead require different boot code that can be parsed and understood by their UEFI firmware.
Luckily, creating a boot disc for such a machine is also fairly straightforward, but it helps to take a step back and understand what exactly is going on.
Fun fact!
The logic to create a standard bootable filesystem image, (typically written to a USB flash drive), and optical disc image, (typically written to a CD), similar to those available as part of the OpenBSD project's official releases is in:
/usr/src/distrib/amd64/iso/Makefile
Enter the El Torito boot specification
Essentially, the extensions described in the El Torito specification allow the BIOS, (or in our case, UEFI firmware), to locate and load an arbitrary number of blocks from the disc, but how this data is interpreted and therefore what we need to write in order to make our disc bootable, is platform dependent.
In the case of the traditional BIOS interface there are actually three modes of operation possible within the El Torito specification. The mode most commonly used in conjunction with the OpenBSD boot code just reads an arbitrary number of bytes, (here 2048), into memory at a fixed address and executes them in X86 real mode. This is similar in principle to BIOS booting from a hard disk where the 512 bytes of the MBR are read and executed, and in fact by default even the load address is the same.
When UEFI firmware boots from an optical disc, it expects to interpet the boot blocks as a FAT filesystem. Within this filesystem it looks for binary boot code in the same place that it searches in the EFI system partition when booting from a hard disk, which in the case of the amd64 architecture is /efi/boot/BOOTX64.EFI.
Note that the data pointed to by the El Torito boot records in this case is simply a raw FAT filesystem. It is not a hard disk image, and there is no GPT or MBR partitioning scheme.
This is in contrast to booting a UEFI based machine from a hard disk which would usually have a GPT with entries for the EFI system partition as well as one or more other partitions depending on the installed operating systems.
It's also in contrast to the hard disk emulation mode of operation which can be used with BIOS booting of an El Torito boot record. In that configuration, the boot image usually would include an MBR.
Exploring the boot catalog
Unsurprisingly, just writing a correctly prepared FAT filesystem to the optical media and pointing the El Torito boot record to it instead of amd64/cdbr isn't enough to make a disc that is bootable with X86 UEFI firmware.
Caveat!
At least in theory it shouldn't be, although in practice there are machines that will boot from such a disc.
We'll return to this issue later on.
To understand why we need to do more than just swap out the old binary bootcode blob for a FAT filesystem containing EFI bootcode, we need to delve a bit in to the El Torito specification and see how it works. Specifically we need to explore the concept of the boot catalog.
The boot catalog is essentially a list describing the size, location and attributes of one or more different blobs of boot code that are present on the disc.
It's a binary file, but the format is fairly straightforward and easy to parse. It also has no fixed location on disc, either in terms of physical block address or path name in the filesystem. However /boot.catalog is a common convention, and the official OpenBSD installation images tend to put it in the architecture-specific directory, such as amd64/boot.catalog.
If a disc includes valid El Torito records at all, then by definition the location of the boot catalog must be stored within a specific type of entry in the set of volume descriptors, present on all ISO-9660 compliant discs. Specifically, there must be a volume descriptor with type code 0x00 at block 0x11. In the case of multi-session discs, the last session is considered for this purpose.
This is how the BIOS or UEFI firmware can find the boot catalog.
A quick look at volume descriptors
The first volume descriptor in an ISO-9660 compliant disc image is always at block 0x10, and it's always of type primary volume descriptor.
We can look at the data contained in the primary volume descriptor from the command line using dd and hexdump:
Typical boot record volume descriptor at block 0x11.
The four bytes at offsets 0x47 to 0x4a contain the block number of the start of the boot catalog, in little-endian format.
So in this case the boot catalog can be found at block 0x1019.
Parsing the boot catalog data
Now that we know how to find the boot catalog on a compliant disc, we can better understand the different requirements for UEFI firmware booting as compared to BIOS booting.
A typical boot catalog for a disc mastered with one of the invocations of mkhybrid or mkisofs that we saw at the beginning of this article might look like this:
Remember that in this example the only boot code that we wrote was the traditional x86 boot code that is suitable for BIOS booting.
For BIOS booting, the second stage bootstrap code cdboot also needs to be present in a suitable location on the disc.
This can be either the root of the CD or as /version_number/architecture/cdboot. These paths are defined in sys/arch/amd64/stand/cdbr/cdbr.S as loader_paths, (and could, therefore, easily be modified). EFI booting does not require the cdboot code as the corresponding functionality is incorporated in the main EFI bootloader.
The first 32 bytes of the boot catalog are known as a validation entry.
This not only serves to confirm that the following data does indeed form a valid boot catalog, but also crucially includes the platform ID that the following record, an initial/default entry, applies to at offset 0x01:
This contains various flags, as well as the location and size of the bootstrap code, (in little-endian format), which for X86 BIOS booting can either be directly executable machine code, a floppy disk image, or a hard disk image, and for UEFI firmware booting, should be a raw FAT filesystem.
Source code reference
The source code for mkisofs and mkhybrid defines a structure struct eltorito_defaultboot_entry in the file iso9660.h to describe this data.
Caveat!
When creating bootable OpenBSD system discs, the byte at offset 0x04 will also typically be set to 0x00 for records that are bootable by an X86 BIOS, and set to 0xef for EFI boot code, just like the platform ID byte.
However this is basically co-incidence, and this behaviour should not be relied on more generally.
In fact, the byte at offset 0x04 is used to hold a system type value. This is intended to be the same as the MBR partition type when a record intended for X86 BIOS booting refers to a hard disk image, so an OpenBSD system would set this byte to 0xA6 in that case.
In reality, since the OpenBSD boot loader is usually used with pure binary boot code rather than a disk image, byte 0x04 is usually set to 0x00, which just happens to match the 80x86 platform ID.
In contrast, the EFI boot code pointed to by an El Torito record is actually a raw FAT partition containing the same boot file as would be found on an EFI system partition. Although there is no MBR partition table, the MBR partition type for an EFI system partition is 0xef and this value is typically used for the system type value stored in byte 0x04 of the initial/default entry.
Creating a disc image that is bootable from UEFI firmware
It should now be obvious why a disc mastered with the commands we saw at the beginning of this article won't, (necessarily), boot on a UEFI system.
The UEFI firmware will be looking for an entry in the boot catalog with platform ID set to 0xef, rather than set to 0x00 as it would be for a BIOS bootable record. If no such entry is found, then the disc is not bootable.
Caveat!
Machines exist in the wild with UEFI firmware that will indeed boot from an entry with the platform ID set to 0x00, and interpret it as a FAT filesystem containing EFI boot code even if the system type value is also 0x00.
Mastering boot discs with such records is strongly discouraged as machines that try to boot the disc in BIOS mode will load the binary data representing the FAT filesystem and execute it as machine code. This will almost certainly result in a crash.
Creating a disc image with an EFI boot code record in the boot catalog, (and no BIOS boot code at all), is straightforward enough, but it does require several steps.
The following commands will do exactly that using mkhybrid:
The choice of 512K for the size of the FAT filesystem to be read by the UEFI firmware is arbitrary, and is sufficient to hold the required files with some space to spare.
The generated boot.catalog should look something like this:
And the resulting disc image should boot on an X86-64 machine with UEFI firmware.
Of course, a boot loader without the operating system installation files is of very limited use, so any practical application would almost certainly want to populate the boot_disc_layout directory with additional files.
Caveat!
If you specify -e with a path that is not in the actual disc image, you get a boot catalog similar to this: