ExoticSilicon.com - using /bin/eject with usb flash drives

Most of you are probably familiar with using the eject command in conjunction with a CD-ROM or other optical device. In this case it physically ejects the loaded disc, either by opening the drive tray or in the case of a slot loading drive just pushing the disc out.

Tray loading drives can, (almost always), be told to insert a disc by using -t argument to the eject command. On the vast majority of optical drives, this closes the drive tray in the same way as pressing the eject button on the front panel.

# eject /dev/cd0c	Eject the drive tray
# eject -t /dev/cd0c	Re-load the drive tray

Portability

The use of the -t option to the eject command to re-load removable media is not particularly portable across operating systems.

It's present function on OpenBSD has been implemented since 2005, so can safely be assumed to be available on any OpenBSD system in the wild.

On other systems, this option might have a completely different function - for example on NetBSD, (where /usr/bin/eject is implemented as a separate utility to /bin/mt), -t specifies the device type, and the -l option, (for load), is used instead.

Before relying on a particular behaviour in portable scripts, consult the eject manual page or source code for the operating systems that you expect to be running on.

What might not be as obvious is that eject can also be used with removable storage devices such as usb flash drives. In this case, there is almost certainly no way for the device to automatically physically remove itself from the usb socket. Even considering a usb storage device with removable media, such as a memory card reader, it's very unlikely that there is an automatic mechanism to load and unload it.

The first thing to realise is that we can run the eject command against a device which is currently in use, in other words while I/O is being performed on it, or whilst it holds a mounted filesystem.

In this case, the eject, (or more accurately, the off-lining), is scheduled for when the I/O is finished, (or the last mounted filesystem is unmounted).

This has a clear practical use when used in conjunction with media which is actually ejectable, for example when writing a tar archive to a tape we may well want to remove the tape and put it into storage once the operation is completed.

In the case of a script writing a backup to a usb flash drive, it might be useful to ensure that no further writes are, (unintentionally), performed to the device between the backup finishing and the device being removed.

Curiosity

Although pretty much every storage device that plugs in to a usb port can be considered ‘removable media’ these days, not every such device actually presents itself to the sd driver as ‘removable’.

This matters here, as the eject command won't work with ‘non-removable’ devices, (technically, those without the SDEV_REMOVABLE flag set).

The reason for this apparent discrepancy is that, historically, devices such as hard disks were not hot-swappable. The device itself would always be present at the same hardware address once the machine had booted, and until it was powered down.

However, hard disk drives of that era may either have had fixed platters, in which case they would be considered non-removable, or exchangeable disk packs, in which case the media was indeed removable.

On a modern system that supports using usb storage devices, the usb subsystem is almost certainly going to implement hot-plugging of the devices themselves regardless of whether any particular connected device supports interchangeable media.

So basically, there are two different concepts of being removable to consider.

Curious readers may have already tried to reverse the effects of an eject command against a usb flash drive without success:

# eject -t sd7

eject: sd7: Input/output error

Nope. It doesn't work.

Unfortunately, it doesn't make the device accessible again. Instead we're just greeted with an error message.

But why? Clearly this functionality could be implemented, as the device in question is still connected and recognised by the usb subsystem.

To find the answer, and eventually implement the missing functionality, we need to delve in to several parts of the source code.

In fact, the eject command in OpenBSD is just a shortcut for mt. This is a utility originally intended to issue various commands to magnetic tape drives, hence the name. It has since grown to support limited functionality with other storage devices such as optical drives.

Looking at the manual page for mt, we quickly learn that there is a whole wealth of interesting commands that we can potentially send to our devices, and that eject merely sends one of them, specifically the 'offline' command.

Sadly, most of these commands don't have any relevance to devices other than tapes. The mt program is certainly a good example of code from a past era which has to a large extent lost it's original purpose.

Operating systems other than OpenBSD usually implement eject as a separate utility, rather than an alias for mt.

Historically, the -t option was used with mt to specify the device to use. Since this was always a tape device, the use of -t as a mnemonic made sense.

On modern systems, the use of -f to specify the device has become more common. OpenBSD removed support for the -t option from mt in 2003, although other systems such as NetBSD still retain it as an alias for -f.

When support for reloading removable media was added to eject in 2005, the same option letter was chosen for this function. This doesn't cause any conflict with the previous -t functionality of mt, as the -t option is only parsed when the mt code is invoked as eject. However, knowing this helps to avoid any confusion when comparing the source code for mt between systems.

The manual page for mt tells us that eject invokes mt with the ‘offline’ command, but observant readers will have noticed that no mention is made of a ‘load’ command anywhere.

So what exactly is going on it the case of eject -t?

Now that we know that the source for /bin/eject is actually in /usr/src/bin/mt, we can open up /usr/src/bin/mt/mt.c and have a look.

It's actually quite simple. If the program is invoked as eject, then the eject flag is set. If the -t option is provided, then the insert flag is also set. When the insert flag is later checked, it switches between selecting COM_RETEN and COM_EJECT, which are both defined near the beginning of the file in the declaration of struct commands com.

So ejecting with the -t option is treated just like mt retension, and in fact this is noted in the cvs commit notes for revision 1.25 of mt.c.

Whichever command is chosen, and however it's sent, ultimately the mt code just ends up invoking an ioctl on the relevant device driver. It's then up to that individual device driver to implement the particular ioctls it can respond to.

If we want to add support for re-loading usb flash drives after ejecting them, there are two ioctls of interest to us.

These are in the #define's listed as the c_code elements in the struct commands com structure that we just saw in mt.c. The first is MTOFFL, as this is responsible for making the device inaccessible in the first place, and second is MTRETEN, which is what the sd driver will receive when we use the eject -t command against a usb flash drive.

Obviously we will need to add code to implement MTRETEN by reversing whatever actions MTOFFL performs.

The function we need to look at in sd.c is sdioctl, and here we can see that the only one of the mt ioctls which is currently supported is MTOFFL - anything else will just return EIO.

# mt -f sd0c rewind

mt: sd0c: rewind: Input/output error

Hardly surprising.

In the case of MTOFFL, the code just does a quick check that the SDEV_REMOVABLE flag is set, indicating that the drive supports this kind of operation, and then sets the SDEV_EJECTING flag. The SDEV_EJECTING flag is then checked by sdclose, and if it's set then a call is made to scsi_start which eventually leads to the appropriate command being sent to the hardware.

Handy hint!

It might seem a bit un-intuitive to be calling a function called scsi_start when we actually want to off-line a device.

However this is indeed correct, as the same basic SCSI command is used at the hardware level for both starting and stopping a connected device - it's just sent with different parameters.

These parameters are defined in scsi_disk.h, as SSS_STOP, SSS_START, and SSS_LOEJ.

To implement our ‘reloading’ of usb flash drives using eject -t, we basically need to call scsi_start from within the ioctl handler for sd.c, in response to receiving a MTRETEN ioctl request.

This glosses over some of the finer details, but should be enough to make it work as a proof of concept.

Most of our new code can be added to the start of the existing case MTIOCTOP block. This alone isn't enough, though, because near the beginning of the sdioctl function we have a check for the SDEV_MEDIA_LOADED flag being set. If it's not set, then the switch statement that contains case MTIOCTOP is not parsed, because unless the supplied command is one of a few specific commands that can be executed without media already being loaded, the whole function just exits.

So at a minimum, we need to add a case MTIOCTOP to the first switch statement in sdioctl, to enable parsing of the following code which we'll add above the existing case MTIOCTOP in the later switch function:

if (((struct mtop *)addr)->mt_op == MTRETEN) {

scsi_start(link, SSS_START, 0);

scsi_start(link, SSS_START | SSS_LOEJ, 0);

goto exit;

}

During testing with various devices, some required SSS_LOEJ whereas others not only didn't require it but were actually prevented from reloading by including it.

There is probably a more elegant way to deal with this, but for testing purposes we can just call scsi_start twice, first without SSS_LOEJ and then immediately again with it.

Amazingly, this code alone is enough to show that the idea works. Now, if we eject a usb flash device we can re-load it afterwards and read from it again:

# bioctl sd7

sd7: <Mass, Storage Device, \\001 \\001>, serial (unknown)

# eject sd7

# bioctl sd7

bioctl: BIOCINQ: Input/output error

# eject -t sd7

# bioctl sd7

sd7: <Mass, Storage Device, \\001 \\001>, serial (unknown)

It seems to work!

The code above seems to work, but one situation that we've overlooked is both ejecting and re-inserting a device whilst it is being accessed.

In other words, doing something like this:

# mount /dev/sd7d /mnt

# eject sd7

# eject -t sd7

# umount /mnt

# mount /dev/sd7d /mnt

mount_ffs: /dev/sd7d on /mnt: No medium found

Not the desired result.

In this case, the code above isn't enough to ensure correct operation because we don't explicitly clear the SDEV_EJECTING flag. This didn't matter in the earlier example, because the sdclose function reset it for us. But if we want to re-load our media before sdclose gets a chance to run, we need to clear the SDEV_EJECTING flag ourselves.

The following version of the above code does exactly that:

if (((struct mtop *)addr)->mt_op == MTRETEN) {

if (ISSET(link->flags, SDEV_MEDIA_LOADED)) {

CLR(link->flags, SDEV_EJECTING);

goto exit;

}

scsi_start(link, SSS_START, 0);

scsi_start(link, SSS_START | SSS_LOEJ, 0);

goto exit;

}

Much better.

Interesting observation

Although implementing MTRETEN for sd devices was mainly an exercise in exploring how the eject command worked internally, whilst testing the code in conjunction with a memory card reader I did find a potential practical use for it.