ExoticSilicon.com - Console by candlelight, using the framebuffer console at night

“Night mode for the framebuffer console”

Adding new visual enhancements to the framebuffer display, including a blue-filtering mode

From warm to cold

Thirty years ago, you might have had a ‘color temperature’ knob on the front of your brand new high-end Super VGA monitor.

(This would likely have been at work, of course, as most home users could barely afford 16-color EGA.)

Twiddling this control towards ‘warm’ during those late nights at the office probably felt good on the eyes, whereas during the day the crisp, accurate color rendition of ‘cold’ was more productive.

Today we'll see how to re-create this useful effect on the OpenBSD framebuffer console.

Talking about color choices - don't forget that this entire website is available in a choice of TEN FANTASTIC THEMES, so be sure to try them all out and find one that suits you!

Introduction

In recent years, this idea has been picked up, recycled, and presented to the consumer as something new. It's no longer marketed as ‘color temperature’, it's now usually, ‘night mode’, or something along those lines.

Just about every new electronic device with a screen now has it, and plenty of desktop environments have introduced it too.

Of course, the one desktop environment of note that has been sorely lacking any kind of software-based color temperature correction is the framebuffer console in OpenBSD.

Until now, that is...

A quick side note:

In some contexts, the term ‘night mode’ is used just to mean a light-on-dark color scheme, which is absolutely not what we're talking about here.

In the true spirit of 21st century recycling, it seems that even this name seems to have been recycled to refer to more than one concept!

In today's presentation...

We could just present this all neatly wrapped up as a short kernel diff and some usage instructions...

But where's the fun in that?

Instead, today we'll take a look at and de-mystify the actual process of coding all of this new functionality.

So if you're a seasoned C programmer who just isn't yet very familiar with the kernel internals, now is the perfect time to learn:

Stuff to learn today

How to add a new sysctl knob to the kernel and parse it's value.
How to add a new escape sequence to the terminal emulation code.
How to adjust the device color palette on the fly.

Wow, all that and a bag of chips sounds too good to be true, huh?

Well, if you like this content from Exotic Silicon, and want to see more then please show your appreciation by linking to us from reputable websites, and mentioning us on social media.

A set of patches to implement the new functionality can be found near the bottom of this page, for any impatient readers who just want to see the end results.

First steps - testing a new palette

Our first goal to is create a new color palette with a warmer hue, by reducing the blue content. If you've read our reckless guide, you'll already know that the color palette is defined in sys/dev/rasops/rasops.c. If we just wanted an alternative palette all the time, we could simply change the RGB values defined there. However, we want a choice of at least two palettes, the normal one and a warmer one.

At this point comes our first design decision. Should we manually define an additional set of sixteen warm colors, and then simply add sixteen to the index of the chosen color when we want to reach into our second palette? Or should the alternative colors be somehow derived mathematically on the fly from the original palette?

Both approaches would be reasonable, especially as there are a large number of unused entries in the rasops_cmap array that holds the RGB values. However, it seems more straightforward and practical to generate the new palette automatically, so we'll use this method here.

Unsurprisingly, there isn't really much experimentation to be done in order to find a suitable algorithm for reducing blue content. Simply shifting the 8-bit blue value right by one bit will reduce the intensity of the blue channel, and in practice this does indeed produce a visually pleasing effect.

We can test this by adding a few lines to sys/dev/rasops/rasops32.c, (assuming, of course, that we are using a 32-bit non-byteswapped display, which most users of modern desktops or laptops will be).

In this case, the variables f and b will contain the 24-bit RGB values for the foreground and background colors respectively. All we need to do is to shift the lower eight bits right by one bit.

/* Reduce blue component */

int blue;

blue=(b & 0xff);

blue=(blue >> 1);

b=(b & 0xffff00) | blue;

blue=(f & 0xff);

blue=(blue >> 1);

f=(f & 0xffff00) | blue;

This logic can be expressed more compactly in C as:

b=(b & 0xffff00) | ((b & 0xff)>>1);

f=(f & 0xffff00) | ((f & 0xff)>>1);

A quick side note:

If you do any experiments of your own within the rasops code, be careful about adding printf calls for debugging purposes. This will often panic the kernel, as certain code paths then become called recursively.

After re-compiling the kernel and re-booting, we're greeted with the effect that we were hoping for.

Assembly optimisation interlude for curious readers

The bit shifting calculation above is actually much easier in X86 assembler than it is in C.

This might sound surprising, but as any elite hacker with a firm grasp on X86 assembler should know, we can just do:

asm volatile ("shrb %%al;" : "+a" (b) : :);

asm volatile ("shrb %%al;" : "+a" (f) : :);

For those readers who are not familiar with X86 assembler, the way this works is that we load the entire 32-bit value into a 32-bit register, %eax, but then perform the right shift only on the 8-bit register that corresponds with the lower eight bits of %eax, which is %al. The final 32-bit value is then read out from %eax again.

This is typically even more efficient than the code produced by an optimising compiler.

To demonstrate this point, consider the following short C program:

int main()

{

int rgb=0x00ffffff;

rgb=(rgb & 0xffff00) | ((rgb & 0xff)>>1);

}

Compiled with clang version 11.1.0 on OpenBSD 7.0-release, and disabling optimisation by using -O0, we get the following assembly output for the bit operations:

movl $16777215, -4(%rbp) # Store 0xFFFFFF in ‘rgb’

movl -4(%rbp), %ecx # Copy ‘rgb’ to ecx

andl $16776960, %ecx # ecx & 0xffff00

movl -4(%rbp), %edx # Copy ‘rgb’ to edx

andl $255, %edx # edx & 0xff

sarl $1, %edx # edx >> 1

orl %edx, %ecx # ecx = ecx | edx

movl %ecx, -4(%rbp) # ‘rgb’ = ecx

With optimisation enabled, things are slightly more complicated. Since our computed value is not actually used, the compiler will simply optimise it away completely. Even if we make use of the value by adding a final call to printf, the compiler will still optimise away the actual calculation, because all of the arguments are constants, so the result can be computed at compile time as 0xFFFF7F.

Changing our code to call arc4random_uniform to set the initial value of variable rgb, forces the compiler to produce code to do the bit shifting at runtime. Compiled with optimisation level -O3, we get this:

movl %eax, %ecx # Copy the value returned from arc4random_uniform from eax to ecx

andl $16776960, %ecx # ecx & 0xffff00

shrl %eax # eax >> 1

andl $127, %eax # eax & 0x7f

leal (%rax,%rcx), %esi # esi = (rax + rcx)

Huh? What's leal?

Note the use of the lea opcode, ‘Load Effective Address’, to perform the logical ‘or’ operation. This might be un-intuitive for readers with a background in assembly coding on non-X86 platforms. The lea set of opcodes, of which leal is one, is intended for calculating addresses by doing addition and bit shifting. But since the lea family of opcodes doesn't actually load anything into the calculated address, we can use leal as a convenient general purpose addition and bit-shifting instruction. An optimising compiler typically will use the lea instruction for these sorts of operations.

However, with our hand-grafted in-line assembly, the compiler produces the following:

movl $16777215, %eax # Store 0xFFFFFF in eax

shrb %al # eax=(eax & 0xffff00) | ((eax & 0xff)>>1);

Which is clearly much better.

Fun observation:

It's worth noting that despite this code being run for every character painted to the screen, the performance hit is almost un-measurable, especially with the assembler version.

A quick color test chart

Since a large amount of the output on the console is by default white text on a black background, if we're going to be making adjustments to the color rendering it would be useful to have a program that outputs some sort of color test chart.

TEST TEST TEST TEST TEST TEST TEST TEST

Although userland programs should usually consult the terminfo database to obtain suitable escape sequences for displaying color on the current output device, in this case we can happily put the appropriate escape sequences directly into the code, as this program is specifically intended to display the color test chart on the OpenBSD framebuffer console, (in it's vt100 emulation mode). This also ensures that the program will produce the correct output even if the wrong terminal type is selected.

These are the escape sequences we'll be using:

␛[0m Reset all attributes to default values.

␛[1m Set bold mode, (displayed as high intensity).

␛[3Xm Set foreground to color X, (ranging from 0-7).

␛[4Xm Set background to color X, (ranging from 0-7).

A quick side note:

In this article, ESCAPE, 0x1b, is represented with the following glyph: ␛

In most shells, to enter the escape character it's necessary to insert the ‘literal’ control character first. This can usually be done by typing control-v. A following ‘escape’ will then typically be displayed on the console as ^[.

The above escapes sequences all immediately follow this with a further regular [ character.

We can create the color chart quite easily with the following short shell script:

#!/bin/ksh

for i in 0 1 ; do

echo -n ␛["$i"m

for b in 0 1 2 3 4 5 6 7 ; do

for f in 0 1 2 3 4 5 6 7 ; do

echo -n ␛[3"$f"m␛[4"$b"m' TEST '

done

echo "␛[49m␛[39m"

done

This chart can be used not only to evaluate the visual esthetics of an alternative color palette, but also to check that different combinations of foreground and background colors remain visibly distinct.

Moving the code to a better place

Before implementing a new sysctl to allow us to activate and deactivate the alternative color palette whenever we want to, we should consider whether there is a better place to put our code than in the middle of the rasops32_putchar() function. This was convenient for a quick test, but the extra code is running every time a character is drawn, so we are performing the exact same calculation repeatedly. Although performance is very good with this specific single bit shift of the blue value, if we later decide to expand our code to include other effects then the extra CPU load might become noticeable.

Also, if we wanted to support other color depths we would need to patch the corresponding functions in each of rasops24.c, rasops15.c, and so on.

The various bit-depth specific functions work with a device color map, rather than the raw RGB values which are defined at the beginning of rasops.c. The device color map is computed once during initialisation of the display, in the function rasops_init_devcmap(), and then stored for re-use.

This means that we could insert our color-modifying code in the rasops_init_devcmap() function, and then call it to re-calculate the device color map whenever we change the palette via our new sysctl. Doing this would completely eliminate the per-character overhead.

Device color maps

Whereas the color map values in rasops.c are defined as 24-bit RGB values, in the device color map color values are stored in the format that the hardware expects them to be in. This greatly simplifies and speeds up the process of plotting each pixel that is required for every character drawn.

In the case of 32-bit color, the transformation is usually straightforward, as the only difference between the raw 24-bit values and the device color map is an extra byte of padding to align the data for each pixel to a 32-bit boundry:

32-bit RGB

0x00000000RRRRRRRRGGGGGGGGBBBBBBBB

A 15-bit display would usually expect the data in this format:

15-bit RGB

0x0RRRRRGGGGGBBBBB

Byte-swapped versions of these formats also exist, but hardware using them is less common:

Byte-swapped 32-bit:

0xBBBBBBBBGGGGGGGGRRRRRRRR00000000

Byte-swapped 15-bit:

0xGGGBBBBB0RRRRRGG

The important point to note is that the bit-depth specific functions such as rasops32_putchar(), only deal with the device color map values, and not the raw 24-bit RGB values in rasops_cmap[].

Looking at the rasops_init_devcmap() function, we can see that bit depths of 1, 4, and 8 bits per pixel use a fixed hardware palette which is not based on the RGB values in rasops_cmap at all. This means that our color changing code will have to be limited to bit depths of 15, 16, 24 and 32 bpp.

The single argument supplied to rasops_init_devcmap() is simply a pointer to a rasops_info structure. This is effectively an opaque cookie to access the parameters of the display in question. We can ignore the first part of the function which deals with bit depths, (as given by ri->ri_depth), of 1, 4, and 8. The next section of code, which creates the device color map for the remaining bit depths, is less complicated than it initially appears.

Variable p is initialized to point to the first byte of rasops_cmap, and will iterate over the red, green, and blue values for each defined color, in other words, each entry in the table that was defined at the beginning of the file. We loop through the 16 defined colors with variable i, and the code within that loop actually does the job of packing the bits into variable c in the format required by the hardware. The values of ri_Xnum and ri_Xpos, where X is either r, g, or b, determine the number of bits for that channel, as well as their position within the final value.

If we look at rasops.h, we can see a comment saying that if ri_Xnum and ri_Xpos are set to zero, then default values will be applied. Looking in rasops32.c, at the very beginning of rasops32_init(), we can see the code that actually does this. In fact, it only checks ri_rnum, and proceeds to set all six variables to default values if ri_rnum is zero.

However, looking very carefully, we can see that these default values actually set the positions of the red and blue values within the final 32-bit value to be swapped. This seems unusual, as it would produce entries in the device color map in the following format:

32-bit BGR

0x00000000BBBBBBBBGGGGGGGGRRRRRRRR

This BGR ordering is the opposite of the order set up by the graphics drivers in /usr/src/sys/dev/, as can be seen by running the following simple grep command:

# grep -r ri_.pos /usr/src/sys/dev/* /usr/src/sys/dev/rasops/rasops??.c

Since all of the graphics drivers explicitly set values for these variables, the default values will not be used, so this point is somewhat moot. It does seem interesting, though, and might even be a bug.

Adding our color palette changing logic to rasops_init_devcmap()

If we wanted to support all combinations of 15, 16, 24, and 32 bpp hardware, as well as byte-swapped variations, then we would need to add our code to the assignments made to variable c at the beginning of the loop, for each of the red, green, and blue channels.

This is certainly possible, but if we restrict our support to 24 and 32 bpp modes, and ignore byte-swapping, our changes to the existing source code can be far simpler and less intrusive, which is useful if we intend to port our local changes to future versions of the OpenBSD kernel every six months.

To test the concept, we simply need to insert either one of the two versions of the line of bit shifting code immediately before the final assignment of c to the entry in ri_devcmap right at the end of function rasops_init_devcmap().

X86 assembler version:

asm volatile ("shrb %%al;" : "+a" (c) : :);

Generic C version:

c=(c & 0xffff00) | ((c & 0xff)>>1);

So we are modifying the value of variable c, which is already in the format that the hardware is expecting, just before assigning it's value to the entry in the device color map array.

If we now also remove our previous changes to rasops32.c, re-compile and re-install the kernel, then reboot into the newly compiled kernel, we will once again see our yellow-tinted output on the console.

The difference is that our code is now being run just once, at the time of display initialisation, rather than every time a character is painted.

Now all that we need to do in order to have this feature selectable at run-time, is to implement a new sysctl adjustment, and ensure that rasops_init_devcmap() is called when it's value is changed.

Implementing a new sysctl

Readers who are not familiar with the kernel internals might naïvely assume that there is a simple library function that can be called with the name of the sysctl value we want to read, and which will return the result. Alas, things are not quite that straightforward, but nevertheless, implementing a new sysctl is not particularly difficult once you know the procedure.

The manual page for sysctl_int() will give you an idea of the complexity of the sysctl interface, but don't worry if you don't understand it.

Our new sysctl will be ‘kern.exotic’, and it will contain an integer value. This will allow us to select the default color palette by setting it's value to 0, or any number of alternative palettes with other settings.

The first file we need to edit is sys/sys/sysctl.h, which contains the list of identifiers in the kern hierarchy. The last entry in this list should be KERN_MAXID, which indicates the largest number in use. As of OpenBSD 7.1-release, this is set to 90, (the most recent addition was kern.video.record, made during the development cycle of OpenBSD 6.9). We just need to increase the value of KERN_MAXID to 91, and add our own definition as 90:

#define KERN_EXOTIC 90

#define KERN_MAXID 91

Immediately after this, we should add our new sysctl to the end of CTL_KERN_NAMES:

{ "exotic", CTLTYPE_INT }, \

At this point, if we were to re-build the kernel, then the definitions of the new sysctl would be compiled into it. However, we will also need to re-compile the userland utility /sbin/sysctl in order for it to recognise the new sysctl that we've added, and that utility includes the sysctl.h header file from /usr/include/sys/sysctl.h, rather than from it's location within the kernel source, so we need to make sure that our changes are reflected there as well:

# cp -p /usr/src/sys/sys/sysctl.h /usr/include/sys/

Then we can re-compile and re-install /sbin/sysctl:

# cd /usr/src/sys/sbin/sysctl

# make

# make install

With these changes in place, and rebooted into a freshly compiled kernel, we can now access our new sysctl, and change the value stored in it:

# sysctl kern.exotic

kern.exotic=0

# sysctl kern.exotic=1

kern.exotic: 0 -> 1

# sysctl kern.exotic

kern.exotic=1

Making use of our new sysctl

Of course, our new sysctl doesn't actually do anything yet. For that we need to add code to sys/kern/kern_sysctl.c.

The easiest way to make use of an integer sysctl value in the kernel is to create a globally-scoped integer variable which will mirror it's value. In other words, when a new value is set from userland using /sbin/sysctl, that value will update the global variable.

We'll call our global variable ‘exotic’, and define it in sys/dev/rasops.c, at the beginning of the file just after the includes, with a simple:

int exotic=0;

Now, we need to add a couple of lines to the large switch statement, in function kern_sysctl, in file sys/kern/kern_sysctl.c:

case KERN_EXOTIC:

return (sysctl_int(oldp, oldlenp, newp, newlen, &exotic));

Where KERN_EXOTIC is the name we defined earlier in sys/sys/sysctl.h, and &exotic is a reference to our new global variable.

This is about the simplest way to implement a new syctl, and will allow us to set kern.exotic to any integer value, which can then be used to control something elsewhere in our code.

If we wanted to restrict the range of values that kern.exotic could be set to, we could add more code to our new entry in the switch statement, but usually it's just easier to interpret unused values as if they were 0 in our own functions.

Our code to change the actual color values can now be made conditional on the value of the global variable:

if (exotic==1) {

asm volatile ("shrb %%al;" : "+a" (c) : :);

}

At this point, we're almost done. Now we just need to make sure that rasops_init_devcmap() is called after updating kern.exotic.

Calling rasops_init_devcmap() when our sysctl is updated

Currently, rasops_init_devcmap() is only being called once, at device initialisation time, right at the end of rasops_init(). This was fine for our quick test, where we just inserted one line to adjust the color values permanently, but if we want to control the effect via our new sysctl then we'll need to call rasops_init_devcmap() whenever it's updated.

Unfortunately, we can't easily call rasops_init_devcmap() directly from our new sysctl handler in sys/kern/kern_sysctl.c, as we need to pass it a pointer to the correct rasops_info structure. One way around this would be to add a second global variable to act as a flag to show that the value had been changed, and then to check this flag every time rasops32_putchar(), (or the putchar function for any other bit-depth), was called.

This works, but it seems somewhat wasteful to be checking such a flag every time a character is written to the framebuffer.

A reasonable trade-off in terms of code complexity and performance, is to add a call to rasops_init_devcmap() to rasops_eraserows(). This function is called fairly frequently, for example during scrolling, when clearing the screen, or switching to a different virtual terminal, but still usually much less often than the various putchar functions.

Re-initialising the device color map from here does cause a slightly unusual visual effect, though. If the cursor is not already at the bottom of the screen, and so the screen doesn't scroll when the sysctl command is entered, the next line of text will not immediately be displayed with the new palette, since we haven't yet called rasops_init_devcmap() to update it. A simple switch to another virtual terminal and back, or clearing the display will cause the screen to be re-painted using the new palette, though, and the trade off seems worthwhile for the lower performance overhead.

Of course, wherever we call rasops_init_devcmap() from, existing text on the screen will not automatically be re-painted. It will remain on the display in the old color palette until it's re-painted manually, usually either by a scroll, or by switching VTs. If we wanted the whole display to be automatically updated, we would need to re-write the contents of each character cell after loading the new color palette.

So, to call rasops_init_devcmap() from rasops_eraserows(), we just need to add the following line immediately after the assignment to variable ri:

rasops_init_devcmap (ri);

Then re-compile the kernel and re-boot, ready for testing!

Testing the new feature

By default, when we boot into the new kernel we are initially greeted with the standard color scheme.

All we need to do to see the alternative color scheme is to log in as root, set kern.exotic to 1, and clear the screen:

# sysctl kern.exotic=1

kern.exotic: 0 -> 1

# clear

And there we are!

Late night hacking sessions just got more comfortable!

Before and after

OpenBSD/amd64 (local.workstation) (ttyC0)

Adding additional alternative palettes

Now that we have the basic functionality working...

We can easily add some extra alternative color palettes by replacing the if statement with a switch, and simply applying different mathematical transformations to the red, green, and blue values.

This is nicely illustrated by the following block of code:

#define RED ((c & 0xff0000) >> 16)

#define GREEN ((c & 0x00ff00) >> 8)

#define BLUE (c & 0x0000ff)

#define GREY ((int)(((GREEN*0.7)+(RED*0.2)+(BLUE*0.1))))

switch (exotic) {

case 1:

/* Reduce blue component */

#if defined (__amd64__) || defined (__i386__)

asm volatile ("shrb %%al;" : "+a" (c) : :);

#else

c=(c & 0xffff00) | ((c & 0xff)>>1);

#endif

break ;

case 2:

/* Convert to green-scale */

c=(int)(((GREEN*0.7)+(RED*0.2)+(BLUE*0.1)))<<8;

break ;

case 3:

/* Convert to amber-scale */

c=(GREY<<16)|(GREY<<8);

break ;

case 4:

/* Convert to greyscale */

c=((GREY<<16) | (GREY<<8) | GREY);

break ;

case 5:

/* Convert to pink-scale */

c=((GREY<<16) | ((GREY>>1)<<8) | (GREY>>1));

break ;

case 6:

/* Convert to greyscale and reduce blue component */

c=((GREY<<16) | (GREY<<8) | (GREY>>1));

break ;

default:

break;

}

This provides us with a choice of no less than seven different operating modes, when inserted into rasops.c just before the final assignment of c to the device color map array entry in function rasops_init_devcmap():

0 Normal

1 Night mode, (blue light reduced)

2 Greenscreen monitor simulation mode

3 Amber phosphor monitor simulation mode

4 Greyscale mode

5 Shades of pink

6 Night greyscale mode, (yellow tinted greyscale)

A quick side note:

In this example code, we also see how to include architecture-specific code into the kernel by using the C pre-processor to test for architecture specific defines.

Here, we are including the X86 assembly code on the amd64 and i386 platforms, but using equivalent C code on all others.

A side project - adding support for the dim attribute

The framebuffer console on OpenBSD supports a large number of control characters and escape sequences, such as the cursor positioning functions used by vt-100 terminals, as well as ANSI escape sequences for color.

However, one particularly useful escape sequence is missing from the emulation, and that is the dim attribute. Since we're looking at color reproduction on the framebuffer console, we might as well take a few minutes to add support for the dim attribute.

This escape sequence works similarly to the bold attribute, which the OpenBSD framebuffer console renders as high-intensity. For dim text, we'll render the characters at a lower intensity.

␛[2m Set dim mode

The following shell script will output text with four sets of attributes: dim, bold and dim together, normal, and bold:

echo "␛[0m␛[2mDIM"

echo "␛[0m␛[1m␛[2mBOLD AND DIM"

echo "␛[0mNORMAL"

echo "␛[0m␛[1mBOLD␛[0m"

Run on an unmodified OpenBSD system, the output will only show two visibly different intensity levels.

After our changes we will see four intensity levels:

Before and after

DIM

BOLD AND DIM

NORMAL

BOLD

DIM

BOLD AND DIM

NORMAL

BOLD

The actual dimming code

To display a color at half brightness, we can simply shift each of the red, green, and blue channels by one bit, dividing their values by two.

Assuming that we are dealing with 24-bit RGB data, the code would look something like this:

red=((f & 0xff0000) >> 16);

green=((f & 0x00ff00) >> 8);

blue=((f & 0x0000ff));

red=red>>1;

green=green>>1;

blue=blue>>1;

f=((red<<16)|(green<<8)|blue);

This can be written more efficiently as:

f=(f>>1) & 0x007F7F7F;

In this case we are simply shifting a whole 32-bit value one bit to the right, and masking the high bits of each of the lower three bytes to avoid shifting a value from the neighbouring byte into them.

Since the dim attribute can be applied on a character by character basis, this time it actually makes logical sense to put the supporting code in the rasops putchar routines. The above line can be inserted directly in to rasops32.c immediately after the value of variable f is assigned from the device color map array.

Note that we are only dimming the foreground color in the example above, and the background stays at it's usual brightness. Of course, it would be trivial to dim the background color as well, we would just need to do the equivalent bit shift for variable ‘b’.

If we now compile a new kernel with this additional code and reboot into it, we will see all of the text displayed at half brightness...

Adding a new escape sequence to the terminal emulation code

Most of the code to handle parsing of terminal escape sequences is in sys/dev/wscons/wsemul_vt100_subr.c, and this is where we will add our code to recognise ␛[2m.

This escape sequence is one of a family which all begin with the CSI, (control sequence introducer), or ␛[. The function that handles these is wsemul_vt100_handle_csi().

Fun fact!

Although the two byte sequence of ␛[ is by far the most common way to signal CSI to an ANSI compatible terminal, it was actually always intended as a legacy 7-bit compatible equivalent encoding, and the single byte 8-bit code 0x9b was also defined as CSI.

However the 8-bit code never gained much popularity, and as 8-bit extensions of ASCII began to emerge encoding printable characters with the same code it's use became increasingly impractical.

Within the CSI family of escape sequences are a set of SGR, (select graphic rendition), sequences, which all have a common format of the CSI sequence, followed by one or more parameters and ending with ‘m’:

␛[parametersm

Looking through the code for wsemul_vt100_handle_csi(), we can see that it mostly consists of a single large switch statement checking the arguments of the CSI sequence, and that case ‘m’ is indeed commented as SGR. This then leads to a nested switch statement to parse the SGR parameters.

Most of the SGR sequences, with the exception of the reset and color selection sequences, simply set or reset bits in the flags variable. These flags are later parsed by the rasops code in rasops.c, in the function rasops_pack_cattr().

The bits of the flags variable are defined in sys/dev/wscons/wsdisplayvar.h, and all begin with WSATTR_. As of OpenBSD 7.1, only bits 0 - 4 have definitions, leaving us free to use the other bits for our own purposes.

For the dim flag, we'll use bit 5:

#define WSATTR_DIM 32

With this in place, we can add a new case to the SGR switch statement in wsemul_vt100_handle_csi():

case 2: /* dim */

flags |= WSATTR_DIM;

break;

We should also modify case 22, which currently clears the bold attribute, so that it clears our new dim attribute as well, either by adding an additional bitmask operation:

flags &= ~WSATTR_DIM;

Or by replacing the existing code with a combined bitmask for the two flag bits:

flags &= ~(WSATTR_DIM | WSATTR_HILIT);

Passing the new flag bit through the rasops code

The WSATTR_ flags are parsed by the function rasops_pack_cattr() in rasops.c, (or more correctly, whichever function is pointed to by the function pointer ri_ops.pack_attr, which could be rasops_pack_mattr() in the case of a monochrome display).

This function prepares the attr value which is eventually passed to the relevant putchar function in the bit-depth specific rasops code. The attr value stores the index to the background color in bits 16-19, the index to the foreground color in bits 24-27, and three flags in bits 0-2. These flags are completely different from the WSATTR_ flag bit definitions, and don't actually seem to be defined as mnemonics but are instead referenced in the code as magic numbers without much explanation as to what they do.

Bit 0 indicates underlining, whilst bits 1 and 2 indicate that the foreground and background colors respectively are a shade of grey, in other words, that the red, green, and blue components are equal.

We'll use bit 3 to indicate that our dim attribute is active. The existing code re-purposes the variable ‘flg’ from representing the bitmap of flags supplied to the function, to storing the new flags to put in the lower bits of ‘attr’. Since it only has to deal with a single incoming flag being passed onwards, the code simply uses the tertiary operator to set or reset bit 0, based on the value it already contains.

UPDATE 2023

Note that the description above is based on the code in OpenBSD 7.1, and the details have changed in OpenBSD releases since OpenBSD 7.3.

During the development cycle of OpenBSD 7.3, Exotic Silicon submitted various console-related patches to the project including code to simplify the behaviour of rasops_pack_cattr().

As a result of these changes, the WSATTR_UNDERLINE attribute bit is now set directly in attr instead of being indicated by bit 0, and bits 1 and 2 no longer indicate grey values.

Unfortunately, this then destroys the contents of the bit that we are using to store our dim flag, so we either need to re-write this existing code, or simply store our dim attribute bit elsewhere and insert it into ‘attr’ afterwards.

By this point in the function, the ‘swap’ variable is no longer used, so we can re-purpose it to hold our flag temporarily:

swap=((flg & WSATTR_DIM)==WSATTR_DIM ? 1 : 0);

Then add it back after the underline flag code:

flg |= swap;

With these changes, our dim attribute will be passed to the purchar routine as part of the attr parameter.

Making the rasops dimming code conditional on the dim flag

All we need to do now is to make our dimming code in rasops32_putchar() conditional on bit 3 of attr being set:

/* Implement dim if bit 3 of the attribute is set */

if ((attr & 8)==8) {

f=(f>>1) & 0x007F7F7F;

}

Re-compile, then re-boot into the new kernel, and we're done!

Now our small test program above correctly displays the four combinations of normal, dim, bold, and bold with dim.

Implementing strikethrough, double underline, and other features

As we have just shown, implementing the parsing of a new escape sequence and it's corresponding graphical effect is relatively straightforward.

This is especially true if we are only concerned with supporting 32 bpp displays.

The graphical effects for strikethrough and double underline are trivial to implement as they are basically just variations on the regular underlining code.

Italic text could easily be simulated by right-shifting each row of pixel data a suitable number of bits to the right.

Blinking text would be more challenging, as this would require on-going updating of the bitmapped display for characters that had already been painted.

Ready made patches

For those readers who are not interested in learning how this all works and would rather just see the end result, we've produced patch sets implementing the features.

Patch sets are currently available against OpenBSD 7.0, OpenBSD 7.1, and OpenBSD 7.2.

If you've skipped reading the full article...

Note that as well as re-compiling the kernel you will need to re-compile /sbin/sysctl as well, after updating /usr/include/sys/sysctl.h.

Refer to the section ‘Implementing a new sysctl’ above.

File	Changes	Description
dev/wscons/wsdisplayvar.h	+3 lines	Add WSATTR_DIM. Add WSATTR_STRIKE. Add WSATTR_DOUBLE_UNDERLINE
dev/wscons/wsemul_vt100_subr.c	+16 lines	Add dim attribute. Add double underline attribute. Add strikethrough attribute.
dev/rasops/rasops.c	+59 lines ~1 line	Declare new global variable exotic. Set bits in attr representing the new dim, strikethrough, and double underline attributes. Implement six new color palette adjustments. Re-initialize the device color map when calling rasops_eraserows
dev/rasops/rasops32.c	+30 lines	Implement color adjustment for the dim attribute. Implement painting of double underline attribute. Implement painting of strikethrough attribute.
sys/sysctl.h	-1 line +3 lines	Add entries for new sysctl kern.exotic.
kern/kern_sysctl.c	+6 lines	Implement new sysctl kern.exotic.
Notes
Patch format	Unified diff with signature
Patch size	6590 bytes
Patch hash	SHA512 (candlelit_console_patch_7.0.sig) = aXsbCKdjuUOlcK/d4SNIKt4ZI+wCY8LZ8KjGR27P068cBUFPorW5rv0CnMTiyWSQWssV+ckkPJh+FtGpB4i2TQ==
Patch hash	SHA512 (candlelit_console_patch_7.1.sig) = +4GvR/Sw1pSzWoPDt2zAJK4qRbfjfdPMsH9Hdn2hv2wpNQO3wrp+h+IhZoJLmyQXCcdVpXIyVPPCyEUfoJVqGw==
Patch hash	SHA512 (candlelit_console_patch_7.2.sig) = 9doJNMi/vyykRCWSvnJOq++LDPsKwvhok/G429RSAOqnA6MVIsUS5iQ+Q12UGfs8DusC0+VjFKIfZGzqvvIbRw==
Applies to	Three diffs are available, for OpenBSD 7.0-release, OpenBSD 7.1-release, and OpenBSD 7.2-release.
Downloads
candlelit_console_patch_7.0.sig	Kernel patch with embedded signify signature.
candlelit_console_patch_7.1.sig	Kernel patch with embedded signify signature.
candlelit_console_patch_7.2.sig	Kernel patch with embedded signify signature.
color_chart	Shell script to display a color test chart.

THE ABOVE LINKED SOFTWARE IS PROVIDED 'AS IS' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL EXOTIC SILICON BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THE SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Extra features!

In addition to the features explained in this article, the version of the code in the patchsets also implements double underlining and strikethrough!

Code compatibility with NetBSD

Although the examples and commentary in this article have been based on the OpenBSD kernel, the broad concepts can quite easily be applied to NetBSD as well. The two codebases are fairly similar in most of the areas that we have touched, the main caveats being:

The changes we've made to rasops_init_devcmap() in rasops.c will apply almost unchanged to NetBSD as well, since the functions are very similar and even use the same variable names. If we are only interested in supporting 32 bpp displays, the changes are actually trivial to make in NetBSD, as we can just put our code to modify the value of 'c' at the end of the case 32 condition of the switch statement.
Adding sysctls is somewhat different.
The individual rasops??.c files are quite different, as the putchar routine is genericified and included in rasops_putchar.h, rather than being written separately for each bit-depth.
Underlining uses two values, ri_ul.height, and ri_ul.off, to set the height and offset of the underline based on the font size. In OpenBSD, these values are hard-coded.
The equivalent function in NetBSD to the rasops_pack_cattr() function in OpenBSD is rasops_allocattr_color(). This function is very similar to rasops_pack_cattr() in OpenBSD, but doesn't set bit 1 of attr for underline. Instead, it passes through bits 0 - 11 from the supplied value for flg, including bit 3 which is defined by WSATTR_UNDERLINE.
As noted above in the section ‘passing the new flag bit through the rasops code’, the code for rasops_pack_cattr() has been changed in OpenBSD since this article was originally published, and the behaviour now more closely matches that of NetBSD.

Summary and suggested programming exercises

In this article, we've seen how to implement a new sysctl knob in the OpenBSD kernel and use it to select between a choice of several different color palettes for the framebuffer kernel. We've also looked at the kernel code that parses terminal escape sequences, and added a new one.

Plenty of scope exists for making further enhancements to the code. Some ideas for additional features include:

Implementing additional alternative color palettes.
Implementing more attributes such as blinking, and italics.
Expanding support for bit depths other than 32 bpp.
Implementing direct selection of 24-bit RGB colors via escape sequences.