An easy start - changing the keyboard layout
Obviously, changing between the fifty or so standard international keyboard layouts already supported by the OpenBSD kernel is fairly straightforward. We can just use the kbd utility interactively, or store our preferred keyboard layout in /etc/kbdtype for automatic configuration at boot time. For most users, this is done during the initial installation.
However, today we're going to start by seeing exactly how keyboard layouts really work in OpenBSD, and what we need to do in order to create our own layout or to truly customise an existing one.
Keyboard layouts are defined separately for different types of keyboard device, such as PS/2 and USB. On the amd64 architecture, these are the two types that we'll mostly be concerned with. The layouts for PS/2 keyboards are stored in /usr/src/sys/dev/pckbc/wskbdmap_mfii.c, and those for USB keyboards are stored in /usr/src/sys/dev/usb/ukbdmap.c.
Since most if not all of the layouts for PS/2 keyboards are also applicable to USB keyboards, the ukbdmap.c file is created automatically from wskbdmap_mfii.c by an awk script in /usr/src/sys/dev/usb/makemap.awk. This can be run by executing ‘make’ in the /usr/src/sys/dev/usb/ directory, and this should be done after editing wskbdmap_mfii.c and before compiling the new kernel, if you expect any changes from wskbdmap_mfii.c to apply to USB keyboards as well.
The format of wskbdmap_mfii.c is fairly simple and intuitive. Each keyboard map is basically just an array of values of type keysym_t, which is defined in /usr/src/sys/dev/wscons/wsksymvar.h as a u_int16_t. These map keycodes which usually represent a physical key in a particular location on a keyboard, with keysyms, which are either characters, control characters, modifiers such as shift and control, or ‘commands’, which cover things like the hotkeys to switch virtual terminals, access the scrollback buffer, and so on.
If you're familiar with the utility xmodmap, used to change keyboard layouts in the X window system, you'll see some vague similarities between the expressions that it uses and the format of the tables in wskbdmap_mfii.c.
Whilst the keycodes are referred to numerically, the keysyms are given verbose names beginning with ‘KS_’. These keysym definitions are in /usr/src/sys/dev/wscons/wsksymdef.h. Usually we don't need to concern ourselves with the actual numerical values for the keysyms, but if we ever want to create a brand new keysym then we would need add it to this file and assign it a unique and unused value. The wsksymdef.h file also defines values for keyboard encodings, (which usually represent country or region specific keyboard layouts), beginning with ‘KB_’, and variants on those encodings, (which represent things like swapping caps lock with control), also beginning with ‘KB_’. The numerical values for the encodings and variants are chosen so that they can be or'ed together to create a unique code for an encoding and variant pair. Adding a completely new keyboard layout would require us to define a new ‘encoding’ value in this file, too.
Those values for keyboard encodings and variants are used at the end of /usr/src/sys/dev/pckbc/wskbdmap_mfii.c, after the main keyboard layout definitions, to create the final big lookup table pckbd_keydesctab based on all of this information. That table is then used by the wskbd device driver so that it can provide keysyms to the kernel. For USB keyboards a similar process happens at the end of /usr/src/sys/dev/usb/ukbdmap.c, and the table ukbd_keydesctab is created.
Keyboard map file structure
Looking at wskbdmap_mfii.c, we can see that the common US layout at the top of the file is used as a base for most of the other layouts. Non-US keyboard layouts which don't have any variant flags set simply re-define the keycodes that differ between themselves and the US layout. Variants of national keyboards further re-define the keycodes of the corresponding non-variant national layout.
The definition for each keycode starts with KC(xx), where xx is the numeric value of that keycode. For example, on the vast majority of national keyboard layouts, the escape key is in the top-left hand corner, and assigned the keycode 1. The ‘zero’ key is almost always to the right of the ‘nine’, and assigned keycode 11.
Looking at the definitions for these two keys in the US keyboard map, we can see that it's organised into four columns, the keycode, followed by three keysyms. The first keysym is used if we are processing the key as a ‘command’, (more on that later), the second one is the keysym to be used as the ‘normal’, unshifted key, and the third is the keysym to be interpreted if the key is pressed together with shift.
The definitions for the national layouts are slightly different. These list keycodes in the same way, but then use up to four columns of keysyms. Instead of command, normal, and shifted keysyms, we have the keysym for the normal unshifted key, followed by the keysym for the shifted key, then the keysym for when the altgr, (right-hand ALT key), modifier is used, followed by the keysym to be used when both the altgr and shift modifiers are used.
Modifying an existing layout
Let's imagine that you have a keyboard without multimedia keys, so it lacks dedicated keys for the volume up and volume down functions.
We can move these functions to the physical F11 and F12 keys instead, so that pressing F11 decreases the volume, and pressing F12 increases it.
The dedicated keys are defined in the US encoding as keycodes 174 and 176, and are not changed in any of the other encodings. The F11 and F12 keys use keycodes 87 and 88, and are also unchanged in the other encodings, (although the iopener keyboard variant does modify those keycodes to be F10 and F11, and the declk variant changes keycode 87 to be escape).
Since these keycodes remain unchanged in the vast majority of keyboard layouts, we can change them ourselves just once in the definition of the US layout, and our changes should automatically be functional with just about any national keyboard.
Assuming that we have already prepared a kernel configuration file called ‘custom.mp’, and run ‘config custom.mp’ to create the kernel build directory as I explained in
part two of this series, we can compile a kernel with our modified keyboard map with just a few commands:
# cd /usr/src/sys/dev/pckbd
# mv wskbdmap_mfii.c wskbdmap_mfii.c.dist
# sed -e s/\(87\)/\(253\)/ -e s/\(88\)/\(254\)/ -e s/\(174\)/\(87\)/ -e s/\(176\)/\(88\)/ wskbdmap_mfii.c.dist > wskbdmap_mfii.c
# cd ../usb/
# make
# cd /usr/src/sys/arch/amd64/compile/custom.mp
# make
# make install
Creating a new custom keyboard mapping, repurposing F11 and F12 as volume up and volume down keys
In this example, we first re-number the F11 and F12 keys to dummy keycode values of 253 and 254. This is mainly so that we can easily change them again with another invocation of sed, but we could equally well have just deleted those entries. Next we change the volume up and volume down keys to the old keycodes for F11 and F12, run the awk script to rebuild the version of the keyboard mapping for USB keyboards, then build and install the new kernel image.
Note that these dummy values that we have introduced to wskbdmap_mfii.c, will appear commented out in the version of the map for USB keyboards when it is automatically generated.
Booting into our modified kernel, we can observe the volume changing as we press the F11 and F12 keys either by playing some sound, or by using a simple shell script to monitor the mixer output levels.
# while true ; do mixerctl outputs.master ; sleep 0.2 ; done
Monitoring the mixer levels whilst testing the new volume keys
If we replace the original keyboard map by renaming the backup file, we need to remember to both ‘touch’ that file to update the timestamp, as well as re-make the version for USB keyboards:
# cd /usr/src/sys/dev/pckbd
# mv wskbdmap_mfii.c.dist wskbdmap_mfii.c
# touch wskbdmap_mfii.c
# cd ../usb/
# make
# cd /usr/src/sys/arch/amd64/compile/custom.mp
# make
# make install
Updating the timestamp on the original file, if we restore it
This demonstrates the general principles for customising keyboard layouts in the kernel.
If you are currently using a US keyboard encoding, you can change the definition of any keycode simply by modifying the entries at the beginning of wskbdmap_mfii.c within the definition for pckbd_keydesc_us. If, however, you are currently using a non-US keyboard encoding, you will need to consider both the definition for pckbd_keydesc_us, and that for your current encoding. If the keycode that you want to change is currently unchanged from the US layout, then you can either change it there, or add it to to the definition for your current encoding. If, however, the keycode that you want to change is already listed in your current encoding to modify it from the keysym it's associated with in the US encoding, then you will need to make the change in the definitions for your current encoding, as any changes you make to that keycode in the US encoding will be re-defined anyway.
Finding keycodes for unknown keys
Some keyboards might have keys that produce keycodes which are not mapped to a keysym at all by default. One way to find out the keycodes is simply to add a call to printf() in /usr/src/sys/dev/wscons/wskbd.c, early in the function wskbd_translate(). The integer variable with the name ‘value’, (not really an excellent choice of variable name in my opinion), that is passed to this function contains the keycode.
We can just add a line to print this value to the console:
printf ("Got keycode: %d\n", value);
Logging keycodes from a locally connected keyboard
Warning!
Your keystrokes will be logged!
If we add this debugging code to the beginning of wskbd_translate, it will log all keycodes received from the local keyboard to /var/log/messages, which is usually set to be world readable. If you log in to the machine using the local keyboard, then your login and password will effectively be recorded in the log file and can be recovered by other users.
Limiting the printf() call to only report keycodes which don't produce a valid regular keysym will likely mitigate this issue. We can mostly do this by adding it within the if statement that checks for res == KS_voidSymbol. Note that modifiers will also return KS_voidSymbol in this test, and will therefore continue to be logged.
Obviously, if you are using a scratch machine that is under your exclusive control, or alternatively are only using the local keyboard for testing keycodes and are logging in via the network or serial console to otherwise operate the machine whilst this kernel patch is in place, the logging issue is probably moot.
This will let us see the keycodes that are being received from the keyboard for each keypress. Note that if you are using a USB keyboard, the keycodes that you see will be different to the ones that are listed in wskbdmap_mfii.c, because as we mentioned earlier, the keycodes for USB keyboards are different to those for PS/2 keyboards. The awk script /usr/src/sys/dev/usb/makemap.awk which creates /usr/src/sys/dev/usb/ukbdmap.c performs a translation of the two sets of numbers, and you can use the source code as a reference to map the displayed numbers back to the corresponding entries that would need to be placed in wskbdmap_mfii.c.
If you have a USB keyboard which produces a keycode that is not present in the conversion table contained in the awk script, and therefore has no equivalent for you to place in wskbdmap_mfii.c, then you will either need to modify the conversion script and add an extra entry to the table, or otherwise edit ukbdmap.c directly, remembering that any changes you make to ukbdmap.c will be overwritten if you ever run the conversion script again. This might be acceptable if you never use PS/2 keyboards, and prefer to work directly with the maps for USB keyboards in ukbdmap.c.
As I mentioned earlier, ‘commands’, are basically key combinations that invoke special behaviour in the kernel. A simple example is switching between virtual terminals, using the combination of control, alt, and a function key. This functionality is implemented by various routines in wskbd.c.
Within the wskbd_translate() function, there is a test to check whether the pressed, (or released), key has a command associated with it. If it does, then internal_command() is called to process it. This happens whether or not the correct modifiers are being pressed, because the modifiers are checked from within internal_command(). In fact, the modifier keys themselves are implemented as commands and processed by the same function.
Of course, if standard keyboards had plenty of unused keys, then it would be simple to set one of them as being a ‘command’ modifier. Then, in conjunction with the function keys, escape, page up and page down, we could change between virtual terminals, enter DDB, and access the scrollback buffer. Unfortunately, most standard keyboards traditionally haven't had unused keys available for this, and we can't just re-purpose another single key for it without losing it's functionality elsewhere, so another method is necessary.
Instead, two keysyms are defined, KS_Cmd1 and KS_Cmd2, by default KS_Cmd1 is bound to both of the control keys, and KS_Cmd2 is bound to both of the alt keys. When used independently, control and alt have their normal functions, and don't invoke commands. However, when both KS_Cmd1 and KS_Cmd2 keysyms are detected together, command mode is activated. So any combination of either control key with either alt key will set the correct bits and allow us to use this functionality.
A third keysym, KS_Cmd is also defined in wsksymdef.h, and this will activate command mode on it's own. It's not bound to a keycode in any of the standard keyboard mappings, but we can easily map it to the ‘menu’ key found on many modern keyboards to the left of the right hand control key. This usually has keycode 221, and is defined in the US keyboard map as keysym KS_Menu.
Since KS_Menu doesn't appear anywhere else in wskbdmap_mfii.c, we can use sed to do a global replace of KS_Menu for KS_Cmd, and then re-compile the kernel again:
# cd /usr/src/sys/dev/pckbd
# mv wskbdmap_mfii.c wskbdmap_mfii.c.dist
# sed -e s/KS_Menu/KS_Cmd/ wskbdmap_mfii.c.dist > wskbdmap_mfii.c
# cd ../usb/
# make
# cd /usr/src/sys/arch/amd64/compile/custom.mp
# make
# make install
Re-purposing the menu key as a single command key
Booting into a kernel with the above modification, we can now switch virtual terminals using the menu key and one of the function keys. The original method of using control and alt together obviously still works, as we haven't removed any of the associated code or key mappings.
However, some keyboards do have additional function keys which generate unique keycodes. With such a keyboard, it might be nice to switch virtual terminals with just a single keypress and not have to use any modifiers.
We can indeed do this, if we modify wskbd.c...
Fun with the numeric keypad
Since most readers won't have a keyboard with, say, F13 - F24 keys that we could use for switching virtual terminals without the need for a modifier key, we'll re-purpose the numeric keypad instead!
This should work on almost all desktop keyboards.
The idea is to bind the same keysyms for commands to the keypad keys as we do to the function keys, but to modify the code in wskbd.c to act upon the commands without any modifiers being pressed at the same time. So we should be able to press ‘1’ on the keypad, and get the same virtual terminal that we get when we press control + alt + F1. Likewise with ‘2’, and control + alt + F2, and so on.
The keycodes for the numeric keypad number keys are not assigned sequentially, (at least not with the PS/2 keycode mapping, they are indeed sequential with the USB keycode mapping, but that's generated automatically so we don't edit it directly).
Nevertheless, adding the necessary keysyms to wskbdmap_mfii.c is easily done either manually, or using a simple shell script that calls sed in a loop:
# cp wskbdmap_mfii.c.dist in
# let kc=71
# for ks in 6 7 8 X 3 4 5 X 0 1 2 9 ; do if [[ $ks != "X" ]] ; then sed -e s/\($kc\),\␉\␉/\($kc\),\ \ KS_Cmd_Screen$ks,/ in > out ; mv out in ; fi ; let kc=$kc+1; done
# mv in wskbdmap_mfii.c
Adding keysyms from the KS_Cmd_Screen set to the keycodes for the numeric keypad keys
Note the tab characters, (␉), in the regular expression!
Prefix these tabs with the ‘literal’ control character control-V when entering them on the console.
If we compile a kernel with this new mapping, the numeric keypad can be used to switch virtual terminals, but only in conjunction with the command modifier.
Obviously, we don't want to disable checking for the command modifier universally, or even disable checking for it only when the keysym is one of the KS_Cmd_Screen keysyms. If we did that, then the function keys would also respond to their commands when pressed alone, and we wouldn't be able to use them as regular function keys.
The code that checks whether the command modifier is being pressed is in the function internal_command(), in wskbd.c. Reading the code for this function, we can see that it first checks to see if the keycode it's processing is one of the command modifiers themselves. If so, then it's status is updated. Next we have the code that implements access to the scrollback buffer. This doesn't require the command modifier to be active, but instead checks that at least one of the shift modifiers is set. If we changed MOD_ANYSHIFT to MOD_COMMAND here, and were running a kernel with the changes described in the previous section, we could use our menu key together with page up and page down to activate the scrollback buffer instead of using shift.
Next we have code to handle keys that modify the screen brightness, and then we have an if statement which checks if either MOD_COMMAND is set or the combination of MOD_COMMAND1 and MOD_COMMAND2. If neither is set then the function returns, and the following code, which includes the logic to switch between virtual terminals, is not executed.
All we need to do to enable command processing for the numeric keypad keys without any modifiers, is to add an expression to this if statement. The value we need to test is ksym2, and not ksym. Both ksym and ksym2 are parameters provided to the internal_command function, and they represent the keysyms that we defined in different columns of wskbdmap_mfii.c. The value of ksym will be the command keysym, and the value of ksym2 will be the regular non-command keysym. For example, when the keypad ‘1’ key is pressed, ksym will contain KS_Cmd_Screen0, whereas ksym2 will contain KS_KP_End.
Checking the actual numerical keysym values of the various keypad keys in wsksymdef.h, we can see that the ones we are interested in are allocated between 0xf295 and 0xf29f. We could just add a check for this range to the if statement:
!(ksym2>=KS_KP_Home && ksym2<=KS_KP_Delete)
Code to test for a keysym not being in the range 0xf295 - 0xf29f. We can add this to the test that decides whether to process a keystroke as a command or not.
This works, but there is a better way to do it, using the KS_GROUP macro defined in wsksymdef.h:
!(KS_GROUP(ksym2)==KS_GROUP_Keypad)
A better way to test for keysyms that represent keys on the numeric keypad.
And now it works as we intended. The keypad keys switch between virtual terminals without the need for any additional modifier keys, and the behaviour of the regular function keys is unchanged. If they are pressed together with a command modifier, they also switch between virtual terminals. Otherwise, they send their regular keysyms.
That's enough for this week, happy keyboard hacking until next time!
This week, we've seen how to modify existing keyboard layouts for both PS/2 and USB keyboards, and how to discover the keycodes for unrecognised keys. We also looked at how the wscons code processes internal commands.
Get ready for next week when we'll be looking at something slightly different, in the form of the softraid code.
IN NEXT WEEK'S INSTALLMENT, JAY WILL BE LOOKING AT THE SOFTRAID CODE. DON'T BE LATE!