ExoticSilicon.com - writing an encrypted CD-R

Introduction

As mentioned in the opening text above, we previously showed you how to use softraid volumes on read-only disk devices.

Casual readers might be expecting something similar this time, however in reality there is almost no technical similarity in the process that we are going to look at today compared with that described in the article linked above.

The reasons for this should be immediately obvious for anyone familiar with the two types of mass storage devices. In the case of write-protected disk devices, the softraid volume has already been, (or at least can be), created in the normal way whist the the drive was, (or is), configured for full read/write access. The issue there was adding the required functionality to the OpenBSD softraid subsystem to cope with not being able to open the device read/write on future attempts, and also to communicate the read-only flag from the softraid volume handling code back to the scsi disk driver.

In contrast, attaching a softraid volume contained on a read-only optical disk, (and handled by the scsi cd driver), isn't expected to fail even without the kernel patches we previously developed. This is because sr_meta_probe() ultimately calls VOP_OPEN() to get raw access to the volume, and as noted in the manual page for VOP_OPEN(), the checking or lack thereof of file access permissions, is dependent on the underlying filesystem, (which in our case doesn't really exist, as we're opening a raw device).

Instead, we need to contend with pre-mastering the volume that will eventually be written to the WORM disc by using a vnd device, and also deal with the fact that differing sector sizes need to be taken into consideration since disklabel structures, (at least on OpenBSD), represent offsets and sizes as block counts.

So there are some subtle but important details to be aware of in order to make it work correctly, but keep reading and you can learn how it's done!

Fun fact!

No special tools required

Creating CD-R discs with one or more softraid partitions, (yes, you can have more than one!), doesn't require any tools beyond what is in the base installation of OpenBSD.

Technically, we're not even limited to crypto volumes, although the practical use of stripe sets and mirrors on optical media is probably almost non-existent.

Disk format requirements

The softraid subsystem expects to find it's data in disklabel partitions of type FS_RAID.

This is hard-coded in various places, as can be seen with a simple search of the source tree:

# grep -R FS_RAID /sys/

The softraid code explicitly checks for the partition type FS_RAID.

The upshot of all this is that we can't write a raw softraid volume directly to optical media without any partitioning and then expect to attach it using the ‘c’ partition of the cd0 device in bioctl:

# bioctl -c C -l /dev/cd0c softraid0

This won't ever work, (at least not without various kernel modifications).

We could write an image of a softraid disk to a file on a standard ISO-9660 filesystem, then mount it as a pseudo disk device using vnconfig for reading. However there would be a performance overhead, and doing so also creates the un-necessary complexity of setting it up every time the disc needs to be read.

Instead, it would be more desirable to simply attach a softraid volume that resides in a real FS_RAID partition on the media.

To do this, we need to write a BSD disklabel to the optical disc.

Creating a disklabel

Actually writing a disklabel to an optical disc is in itself somewhat unusual. Such discs don't usually contain a disklabel on the media at all, and instead a spoofed disklabel is created when the kernel first reads the native filesystem.

The spoofed disklabel of a disc that contains a valid ISO-9660 filesystem always contains exactly two partitions of type FS_ISO9660, (see iso_disklabelspoof() in /sys/isofs/cd9660/cd9660_vfsops.c), so it's useless for indicating a softraid FS_RAID partition.

The following example is a BD-R disc containing a large ISO-9660 filesystem, (it's actually a customised install media disc for OpenBSD versions 7.6 and 7.7, written to a single layer BD-R):

# disklabel -h cd0c

# /dev/rcd0c:

type: ATAPI

disk: CDROM

label:

duid: 0000000000000000

flags:

bytes/sector: 2048

sectors/track: 100

tracks/cylinder: 1

sectors/cylinder: 100

cylinders: 71976

total sectors: 7197584 # total bytes: 13.7G

boundstart: 0

boundend: 7197584

16 partitions:

# size offset fstype [fsize bsize cpg]

a: 13.7G 0 ISO9660

c: 13.7G 0 ISO9660

Spoofed disklabel from a BD-R disc containing a large ISO-9660 filesystem. Note the two identical disklabel entries, partitions a and c.

In the case of a disc which contains an unrecognised filesystem, random data, an encrypted tar archive, or indeed a raw softraid partition, the spoofed disklabel will contain only the ‘c’ partition, which will span the entire recorded space but be of type unused:

# disklabel -h cd0c

# /dev/rcd0c:

type: ATAPI

disk: ATAPI CD-ROM

label: fictitious

duid: 0000000000000000

flags:

bytes/sector: 2048

sectors/track: 100

tracks/cylinder: 1

sectors/cylinder: 100

cylinders: 1311

total sectors: 131074 # total bytes: 256.0M

boundstart: 0

boundend: 131074

16 partitions:

# size offset fstype [fsize bsize cpg]

c: 256.0M 0 unused

A disc containing 256 Mb of raw random data.

Neither of these scenarios provides the required FS_RAID partitions, so instead we need a real disklabel, with real FS_RAID partitions.

Luckily, the kernel will happily read and use a real disklabel on a cd device with no additional configuration necessary at mount time.

Block sizes

There is, however, a detail that we need to be aware of.

Optical discs in common use almost always use a block size of 2048 bytes, whereas vnconfig will by default present an image file as a device having 512-byte sectors.

Since disklabels store offset and size values as sector counts, the same disklabel describes a disk of a different size if it's moved from a 512-byte device to a 2048-byte device.

The tell-tale sign that you're looking at a 2048-byte disc containing a disklabel intended for a 512-byte device is output like this:

# disklabel -h cd0

# /dev/cd0c:

type: ATAPI

disk: vnd device

label: fictitious

duid: 030d411fed495f45

flags:

bytes/sector: 2048

sectors/track: 100

tracks/cylinder: 1

sectors/cylinder: 100

cylinders: 3329

total sectors: 332802 # total bytes: 650.0M

boundstart: 0

boundend: 332802

16 partitions:

# size offset fstype [fsize bsize cpg]

c: 650.0M 0 unused

d: 2600.0M 0 RAID

disklabel: partition d: extends past end of unit

Note the ‘d’ partition is exactly four times the size of the whole disk ‘c’ partition, and /sbin/disklabel even warns that it extends past the end of the unit.

Trying to read such a disc may even cause the device to become unresponsive, for example on the test machine using a SATA connected BD-R drive:

ahci0: unrecoverable errors (IS: 1000000<OFS>), disabling port.

To avoid this problem, we need to add a new entry in /etc/disktab, which defines a disk using 2048-byte sectors. We can then pass this new disk type to vnconfig using it's -t option. The vnd device backed by our image file will report it's 2048-byte sector size to /sbin/disklabel, and the partitions we create will have entries compatible with 2048-byte per sector optical drives that eventually read them.

Fun fact!

The disklabel spoofing code automatically deals with block size

If you've worked with CD image files, such as the official install disc image files from the OpenBSD project, you might be wondering why you've never seen this discrepancy between 512 byte and 2048 byte sector sizes.

After all, in those cases, the same image that can be written to optical media can also be configured as a regular 512 bytes per sector vnode pseudo disk and mounted without the need for a special disk type.

This conveniently works because the code in iso_disklabelspoof() always simply sets the sizes of the partitions it spoofs to the same value as the whole disk size in sectors, thereby completely avoiding any issues with 512 byte vs 2048 byte block sizes.

This value for total device size in sectors is obtained and written to the disklabel by device specific code, (for example, on a cd device it's done in cdgetdisklabel() in cd.c), and this code obviously knows the correct sector size for the device in question.

We can see this effect by comparing the spoofed disklabel from an OpenBSD 7.7 installation image read from an actual BD-R disc, with the same image configured as a vnode pseudo disk:

# /dev/rcd0c:

type: ATAPI

disk: OpenBSD/amd64

label: 7.7 Install CD

duid: 0000000000000000

flags:

bytes/sector: 2048

sectors/track: 100

tracks/cylinder: 1

sectors/cylinder: 100

cylinders: 3820

total sectors: 381920 # total bytes: 745.9M

boundstart: 0

boundend: 381920

16 partitions:

# size offset fstype [fsize bsize cpg]

a: 745.9M 0 ISO9660

c: 745.9M 0 ISO9660

2048-byte sectors

# /dev/rvnd0c:

type: vnd

disk: OpenBSD/amd64

label: 7.7 Install CD

duid: 0000000000000000

flags:

bytes/sector: 512

sectors/track: 100

tracks/cylinder: 1

sectors/cylinder: 100

cylinders: 15276

total sectors: 1527644 # total bytes: 745.9M

boundstart: 0

boundend: 1527644

16 partitions:

# size offset fstype [fsize bsize cpg]

a: 745.9M 0 ISO9660

c: 745.9M 0 ISO9660

512-byte sectors

Note that whilst the overall size is always reported as 745.9M, the cd0c device calculates this as 381920 * 2048 / 1024 / 1024 = 745, whereas the vnd0c device uses 1527644 * 512 / 1024 / 1024 = 745

Creating the 2048-byte block image file

For testing purposes, we'll create a disk image that fits within the capacity of a regular CD-R. This will be divided in to two partitions, one being a softraid crypto volume, and the other a regular ffs2 filesystem.

There is absolutely no requirement to include a second non-encrypted filesystem, and doing so here is purely for demonstration purposes. The whole disc could just as well simply be used for a single softraid volume. The encrypted volume will use a passphrase rather than a keydisk, although there is no technical reason why we couldn't use a keydisk instead.

First, we'll create an empty 192 Mb disk image. The fastest way to do this is using vmctl create, even though the image isn't destined for use with a VM:

# vmctl create -s 192m image_file

This will create a sparse file full of 0x00 bytes.

If you prefer to fill the unused areas with random data, dd can be used instead:

# dd if=/dev/random of=image_file bs=1m count=192

Alternatively, fill the image with with random data.

Next, we configure the newly created file as a vnode pseudo disk, using the disktab entry we added earlier:

# vnconfig -t optical image_file

vnd0

If you don't already have vnd devices configured, it will appear as vnd0.

The following examples assume that the device is indeed vnd0.

Substitute the correct vnd𝓍 if yours is different.

Now we can use /sbin/disklabel interactively to create the two desired partitions:

# disklabel -E vnd0

Label editor (enter '?' for help at any prompt)

vnd0> p m

OpenBSD area: 0-98304; size: 192.0M; free: 192.0M

# size offset fstype [fsize bsize cpg]

c: 192.0M 0 unused

vnd0> a d

offset: [0] 64

size: [98240] 96m

FS type: [4.2BSD] RAID

vnd0*> a e

offset: [49216]

size: [49088]

FS type: [4.2BSD]

vnd0*> w

vnd0> p m

OpenBSD area: 0-98304; size: 192.0M; free: 0.1M

# size offset fstype [fsize bsize cpg]

c: 192.0M 0 unused

d: 96.0M 64 RAID

e: 95.9M 49216 4.2BSD 2048 16384 1

vnd0> q

No label changes.

Creating a RAID partition and a regular ffs2 filesystem partition on the vnd device.

Note that we don't need to use /sbin/fdisk in any way, or write any MBR partitioning to the vnd0 device.

The kernel will find the BSD disklabel that is eventually written to the optical disc without the need for an MBR type A6 partition.

Next, we create the softraid crypto volume on vnd0d:

# bioctl -c C -l vnd0d softraid0

New passphrase:

Re-type passphrase:

softraid0: CRYPTO volume attached as sd8

Now we have a regular sd device to work with.

Then create a partition on the new encrypted disk, and initialize a filesystem on it:

# disklabel -E sd8

Label editor (enter '?' for help at any prompt)

sd8> p m

OpenBSD area: 0-49020; size: 95.7M; free: 95.7M

# size offset fstype [fsize bsize cpg]

c: 95.7M 0 unused

sd8> a d

offset: [0] 64

size: [48956]

FS type: [4.2BSD]

sd8*> w

sd8> q

No label changes.

# newfs /dev/rsd8d

newfs: reduced number of fragments per cylinder group from 12232 to 12032 to enlarge last cylinder group

/dev/rsd8d: 95.6MB in 48952 sectors of 2048 bytes

5 cylinder groups of 23.50MB, 1504 blocks, 6016 inodes each

super-block backups (for fsck -b #) at:

160, 48288, 96416, 144544, 192672,

Adding an ffs2 filesystem partition to the encrypted volume.

Next, we copy the required data to the softraid volume. In this case, we'll just write one text file:

# mount /dev/sd8d /mnt

# echo test data > /mnt/file_1

# umount /mnt

Writing a single text file to the encrypted volume.

Now we can detach the softraid volume:

# bioctl -d sd8

Detaching the softraid volume.

Since we also have the regular unencrypted partition on vnd0e, we'll write a short text file there as well:

# newfs /dev/rvnd0e

/dev/rvnd0e: 95.9MB in 49088 sectors of 2048 bytes

4 cylinder groups of 23.97MB, 1534 blocks, 6144 inodes each

super-block backups (for fsck -b #) at:

160, 49248, 98336, 147424,

# mount /dev/vnd0e /mnt

# echo test data > /mnt/file_2

# umount /mnt

Writing another text file, this time to the regular unencrypted volume.

The vnd0 device can now be unconfigured, and the underlying file written to a CD-R:

# vnconfig -u vnd0

# cdio -v -f /dev/rcd0c tao image_file

Unconfiguring the vnode pseudo disk device, and writing the image it backed on to a physical CD-R disc.

Top tip!

Alternatives to /usr/bin/cdio

Whilst cdio is included in the base system, it only supports writing to CD media, and not other optical formats.

If you are using media that the cdio utility doesn't support, such as DVD-R or BD-R, you'll likely want to use growisofs or cdrecord instead to write the raw image file.

# growisofs -use-the-force-luke=spare:none -Z /dev/rcd0c=image_file

# cdrecord -v -sao dev=/dev/rcd0c image_file

Using the written disc

Now we can check how the disc looks with disklabel:

# /dev/rcd0c:

type: ATAPI

disk: vnd device

label: fictitious

duid: 2a06f21435066b49

flags:

bytes/sector: 2048

sectors/track: 100

tracks/cylinder: 1

sectors/cylinder: 100

cylinders: 984

total sectors: 98306

boundstart: 0

boundend: 98304

16 partitions:

# size offset fstype [fsize bsize cpg]

c: 98306 0 unused

d: 49152 64 RAID

e: 49088 49216 4.2BSD 2048 16384 1534

The offset and size values are correctly expressed in 2048-byte sectors.

Before we can use the ‘d’ and ‘e’ partitions, we need to create device files for them:

# mknod -m 640 cd0d b 6 3

# mknod -m 640 cd0e b 6 4

# mknod -m 640 rcd0d c 15 3

# mknod -m 640 rcd0e c 15 4

# chgrp operator cd0d cd0e rcd0d rcd0e

Creating special device files for ‘d’ and ‘e’ partitions on the first CD device.

Now we can attach the softraid crypto volume directly from the disc, and access the filesystem:

# bioctl -c C -l cd0d softraid0

Passphrase:

softraid0: CRYPTO volume attached as sd8

# mount -oro /dev/sd8d /mnt

# cat /mnt/file_1

test data

Note that we explicitly mount the filesystem on the new sd device as read-only. See the next section, ‘caveats’, for more details on this.

At this point, it's working!

Caveats

It's worth noting that some errors will be displayed on the console:

sd8 at scsibus3 targ 5 lun 0: <OPENBSD, SR CRYPTO, 006>

sd8: 95MB, 2048 bytes/sector, 49020 sectors

softraid0: roaming device vnd0d -> cd0d

cd0(ahci0:7:0): Check Condition (error 0x70) on opcode 0x2a

SENSE KEY: Illegal Request

softraid0: I/O error 5 on dev 0x603 at block 16

softraid0: could not write metadata to cd0d

The first two lines are the expected regular attachment messages for the new softraid volume. However, note the absence of a read-only indicator at the end of the second line. This is because the call to VOP_OPEN() in sr_meta_probe() succeeds, even though it is requesting read/write access.

The third line about roaming the device, (in this case from vnd0d to cd0d), will always appear since there is no way for the metadata to be updated on the write-once media to record this change. Indeed, this is what the rest of the output is reporting, the fact that a write to the device underlying the softraid volume failed.

Nevertheless, apart from flooding the logs somewhat these error messages do not impede the use of the attached volume.

However, since the softraid sd device doesn't report itself as read-only, it's best to explicitly mount any filesystems on it using the -oro option to mount.

If such a filesystem is mounted as read-write, (which would be the default), then whilst reads will succeed, any attempt to write to it, (including, for example, updating file last access times), will cause more errors from the scsi subsystem.

This can even make unmounting the filesystem difficult, as umount will report an I/O error and fail:

# umount /mnt

umount: /mnt: Input/output error

Trying to unmount a filesystem mounted as read/write on an underlying device that is read-only.

To work around this, first update the existing mount to read-only, and then unmount the filesystem:

# mount -u -oro /mnt

# umount /mnt

Succeeds!

Caveat!

Updates to the metadata format

Occasionally, changes are made to the on-disk format of softraid metadata. As of the time of writing, the last such change happened in 2016, with the update to the current version 6.

On a regular re-writable hard disk, the metadata is typically updated automatically the first time it's attached to a system running a kernel version that supports a later softraid metadata format than is encountered on the media.

Obviously this cannot happen on a write-once disc, (which might even be considered an advantage, since once the metadata has been updated, the volume is no longer easily readable on an older system).

A workaround would be to read the entire contents of the disc in to an image file, then configure a vnode pseudo disk to back that image and make it available as a read/write vnd device, which could then be attached to the softraid subsystem and have it's metadata updated. If required, the updated image could then be written back to a fresh disc.