So as we've just said, internally the character values are unsigned 32-bit integers. That's fine as an internal representation, but throwing around 32 bits to represent each character when the majority of English text only uses the first 128 character positions and could be represented with just 7 bits is obviously not ideal. Add to that the practical need to be backwards compatible with regular ASCII encoding, and at least not too incompatible with existing 8-bit encodings that mirror ASCII for the low 7-bit characters and put a variety of different characters in the top 128 slots.

This was the situation faced in the early 1990s, and amazingly, for once, common sense actually prevailed in the IT industry. A sensible, logical, and efficient encoding was developed which gained broad acceptance in the years that followed. This was UTF-8, and in 2023 it's a reasonable bet that any new software that wants maximum compatibility in interchange applications where ASCII or 8-bit national encodings were previously used, is either going to talk UTF-8 or at least not choke on it.

Whilst UTF-8 is a more complex encoding to handle than plain ASCII, compared with the somewhat bizarre and rather badly thought out from a technical point of view encoding known as UTF-1, implementing it is a breeze.

In resume, the idea is that programs can and should use any convenient internal representation of individual characters, or character strings that need some kind of processing done on them. But for data interchange, including sending output to a terminal emulator, the UTF-8 encoding system is usually the best choice. Terminal emulators that previously supported ASCII and various ‘national’ 8-bit character sets with limited sets of accented characters should be updated to correctly interpret UTF-8 data streams.

Certainly. At least that's the intention.

Within the X Window System, things work quite well. Xterm supports UTF-8, and so it's possible to just cat a UTF-8 encoded text file to the console and see the correct output. Assuming that your font of choice actually includes all of the glyphs, you'll see accented letters, mathematical symbols, line drawing characters, and more. Some of the lesser used and more obscure unicode features might be missing support, but the actual decoding from UTF-8 into the correct unicode codepoints works just fine.

But not everybody uses X, or at least not all of the time.

Since the console works in ISO 8859-1 by default, there are escape sequences to switch it in to and out of UTF-8 mode.

The escape sequence to enter UTF-8 mode is ␛%G, and to explicitly leave it we can send ␛%@, although various other escape sequences also have the effect of leaving UTF-8 mode.

This can be demonstrated from the shell. By default we are in ISO 8859-1, so to display ÿ, we can just write an 0xFF byte, (which is 0377 in octal):

Now, if we enter UTF-8 mode, as expected this no longer works.

The 0xFF byte is not a valid character in UTF-8, and should be ignored, (in the context of a terminal emulator - other applications may treat it as a fatal error condition). The correct way to display ÿ at codepoint 0xFF is with the two byte UTF-8 sequence 0xC3 0xBF.

The UTF-8 sequence mostly works as expected:

However, look again at the previous example where we sent 0xFF. Notice the two blank lines after the second echo command before the shell prompt re-appears.

There was also an extra newline in the output when we sent the correct UTF-8 sequence.

In the mean time, further compare the output of the following commands

Now something definitely seems wrong. The output should be the same whether the 0xFF byte is there or not, but it's different.

What's more, it's not just a case of an incorrect glyph being printed - the spurious newline isn't even at the point where the rogue 0xFF has been placed in the datastream.

Here we are seeing a classic case of non-conformant input affecting the behaviour of a program processing the data in an unexpected way.

This is usually a bad thing, and seeing it should cause alarm bells to ring in a competent programmer's mind.

Of course, we're digressing somewhat here and skipping ahead in our investigation. Let's get back to testing the current UTF-8 implementation.

The UTF-8 sequence for the first codepoint beyond the 8-bit range, (I.E. codepoint 256, 0x100, or 0400 in octal), is 0xC4, 0x80.

Since we don't yet have a font with a glyph in this position, we would expect to see a question mark displayed instead. Why? Because sys/dev/rasops/rasops.c includes code to substitute ‘?’ for a character that is not present in the currently selected font. This code is in the function rasops_mapchar.

However, testing shows that this doesn't happen:

Sending the sequence for codepoint 0x100 does not produce the expected result, and the issue with an extra linefeed is also still there.

By the time the codepoint reaches the rasops putchar routines as the fourth parameter ‘uc’, it's basically just a linear offset in to the font data. There really isn't much to go wrong at that point.

Before then, wsemul_vt100_output_normal calls the rasops mapchar function. As we discussed above, if the character is not present in the font it's remapped to a question mark.

Since we're not seeing any output at all, with no glyph painted and not even any cursor movement over the space it should take up, the problem obviously lies in the wscons code and the parsing of the UTF-8 data, rather than in the graphically-orientated rasops code.

This is trivially testable, by inserting one line in sys/dev/wscons/wsemul_vt100.c, right at the beginning of wsemul_vt100_output_normal:

After re-compiling the kernel and re-booting in to the new kernel, sending a backtick to the console, (either with an echo command from the shell or just typing a backtick at the shell prompt), we get ÿ.

Surprised? Don't be. Remember that since we rebooted, by default the console is not operating in UTF-8 mode but in ISO 8859-1 instead. In this mode, the value 0xFFFF is just being logically and'ed with 0xFF, so we end up losing the high-order byte and seeing ÿ.

However, if we switch to UTF-8 mode with ␛%G, then displaying a backtick does indeed send the value 0xFFFF to the re-mapping function, which faithfully re-maps it to a ‘?’ character.

Before we look at the UTF-8 decoding logic in the OpenBSD kernel, we need to understand how UTF-8 encoding actually works. Thankfully, this is quite simple.

UTF-8 is a variable length encoding. A single codepoint can be encoded using one, two, three, or four bytes.

The single byte case is obvious - codepoints between 0 and 127 are coded as a single byte with that value, which will obviously have bit 7 reset.

On the other hand, bytes in a UTF-8 datastream that have bit 7 set, if they are valid at all, will always be something to do with a multi-byte sequence. Furthermore, a multi-byte sequence should only contain bytes with bit 7 set, (and therefore nothing in the 0x00 - 0x7F range).

The first byte of a multi-byte sequence indicates the total number of bytes in the sequence by the number of high order bits that are set to 1, before the first 0 bit is encountered. Any remaining bits after that first 0 bit, encode the highest bits of the codepoint being encoded.

Second and subsequent bytes, (known as continuation bytes), of a multi-byte sequence always have bit 7 as 1, and bit 6 as 0, (a combination that would make no sense as the first byte of a multi-byte sequence), with the lowest 6 bits encoding the next lowest order bits of the codepoint.

The code that implements the multi-byte UTF-8 decoding logic lives in wsemul_subr.c, right at the beginning in the wsemul_getchar function.

When UTF-8 support is disabled or not present in the kernel, this function is quite trivial because there is a one-to-one relationship between bytes input and characters output. As a result, no special decoding logic is required, and data is simply drained from a buffer.

This is not the case when processing UTF-8 encoded data, as UTF-8 is a variable length encoding. A single codepoint can be encoded using one, two, three, or four bytes.

It's quite easy to see how the code is intended to work. The while loop processes characters from the buffer until it either runs out, or we have a valid character. This could be a single byte character, or a successfully re-constructed multi-byte character.

If a character is successfully returned in state->inchar, then the function itself is supposed to return 0. Otherwise it returns EILSEQ.

The first bug is fairly obvious. Looking near the end of the function, we see some checks for bits 5, 4, 3, and 2 being set, and if they are then mbleft is increased. Clearly mbleft is a counter for the number of remaining bytes to process in this multi-byte sequence.

By pure co-incidence, this works for the values encoding codepoints 0x80 - 0xFF, because those codepoints always have bit 8 unset, (because they are 8 bit values which only use bits 0 - 7). Bit 8 of the codepoint would be stored in bit 2 of the first byte, as can be seen in the following table:

Unfortunately, as soon as we try to process a two-byte sequence encoding a codepoint of 0x100 or above, the check for bit 2 being set returns true, and mbleft is increased by one. The effect of this will be that instead of returning the re-constructed value after the second byte, we'll loop again and try to apply the next byte in the buffer to the value of the codepoint that we are building.

We might at this point naïvely expect the following character to be swallowed, but whether due to design or accident, when a continuation byte which should have it's highest bit set doesn't in fact have it set, the code doesn't actually return EILSEQ, but instead just assigns this value to rc and then falls through to the following if statement that checks whether it's a single-byte character or a new multi-byte one.

So effectively, when we tried sending the 0xC4, 0x80, sequence to print character 256, the kernel tried to interpret it as a three byte sequence instead of a two-byte sequence. The trailing newline caused the re-assembly of the multi-byte character to fail, but we fell through to code which then correctly processed the newline character.

Even more interesting things happen if there is no following character. If we output just the 0xC4, 0x80 sequence without a trailing newline using the -n option to echo, then the console will hang. The shell prompt will not re-appear until we press a key, at which point the entire prompt will be printed.

Readers who are familiar with UTF-8 encoding will already have noted that we shouldn't be checking for bits 3 and 2 being set anyway, as these would encode five and six byte multi-byte characters. Such encoding was valid in the original UTF-8 proposal, but has long since been deprecated.

In any case, we should be checking bit 4 if and only if all of bits 7, 6 and 5 are set.

Although there are plenty of fonts available to download from the internet, as well as programs to convert them to the right format to use on the console, for testing purposes I designed some new characters:

These new glyphs can be added to the gallant12x22 font by simply appending the data at the end of the existing declaration for gallant12x22_data[], and changing the value of .numchars from 256 - ' ', to 260 - ' '.

The problem with the code this time is perhaps less obvious. Presumably the function must either be returning EILSEQ, or possibly a control character, because we don't get a glyph output at all when processing these three and four byte characters.

Luckily, reading through the code and substituting in to the variables appropriate values that we might expect to have whilst processing the second byte, (I.E. the first continuation byte), quickly shows up the bug.

Considering codepoint 2048 in UTF-8, we would have the three-byte sequence 0xE0, 0xA0, 0x80, which in binary is 11100000, 10100000, 10000000:

The first byte is processed just fine by our modified code, and will set mbleft to 2.

The second byte has exactly bit 7 set out of bits 6 and 7, so we end up running the ‘else’ clause of the first nested if statement. Here the bits are added to tmpchar, where the ultimate value of the decoded codepoint is being accumulated, and mbleft is decreased.

So far, so good. But now we get to the buggy code.

If mbleft has reached zero, which at this point when processing two-byte sequences it would have done, we set rc to zero to indicate success, and promptly break out of the while loop to exit the whole function, (after writing some locally changed values back to the supplied pointers).

Unfortunately, if mbleft hasn't reached zero, we just fall off of the end of the ‘else’ clause, and continue processing. The same byte is then parsed by the next block of code, the block following the comment about deciding whether it's a multibyte sequence or not.

So what exactly happens if we continue following the program flow through this erroneous code path?

This is an important question to ask when finding a bug in existing code that has been present for a long time on production systems. Rather than just fixing the problem, we need to establish whether the bug is exploitable on machines out ‘in the wild’ in a harmful way or not.

Well in this case, the continuation byte, which has bit 7 set, obviously doesn't match with the next if statement, so it's not treated as a single byte character. The next test checks whether bit 6 is reset, which, of course, it should never be for the start of a muti-byte sequence. Since in this case it is, mbleft and tmpchar get set to zero, and rc is set to EILSEQ to indicate the error condition.

However, we don't exit the function at this point! Instead, we loop around again and grab the next byte, the third and last byte of the sequence. At this point, since mbleft is now zero, we don't even touch the ‘else’ clause of the first nested ‘if’, but instead end up directly back at the test for a 7-bit character or the start of a multi-byte one. The single character test fails, as we have bit 7 set, and we end up running the ‘abort the sequence’ code again, (even though we were not in a sequence anymore, at least according to mbleft being zero), due to bit 6 being reset.

Finally, we end up at the start of the while loop again, where we process the next character in the buffer, (if there is one), normally.

The fix this time is to add a check for still having bytes remaining in a multi-byte sequence, and jumping back to the beginning of the while loop using a continue statement if we do. This avoids running the same byte through the code below and ultimately aborting the multi-byte character that we were constructing:

With the obvious bugs in the UTF-8 decoder fixed, we might naïvely assume that everything would ‘just work’.

This is, of course, ignoring the fact that the UTF-8 decoder is still far from perfect. It still allows overly-long encodings, and doesn't reject the UTF-16 surrogate codepoints, but the point is that well behaved software should display the correct text in the correct place.

Unfortunately, it doesn't.

The following perl program demonstrates the problem by outputting the UTF-8 sequences for characters with codepoints between 192 and 287:

To the trained eye, this is clearly an issue of confusing bytes with characters. Somewhere in the code, our ‘current position’ in a buffer is not being updated correctly. Instead of moving forward by two, three, or four bytes, we're moving forward by one.

Obviously there is more going on than just this, though, because if we were just moving through a buffer and skipping over or failing to skip over the correct number of characters, we'd see missing or repeated output along the way. That is not happening, instead we appear to be outputting the data correctly but somehow repeating the last bit of it.

Seeing this kind of program behaviour suggests that there are, in fact, two, (or more), pointers to the same data that should be being kept in sync but due to a bug are not. Where possible, it's best to avoid writing software like this, but if it is necessary then being diligent and not letting this classic class of bug creep in to your code is absolutely vital.

Just imagine if this same sort of error was to occur elsewhere, such as copying between buffers that contained cryptographic keys. We could potentially write past the end of the target buffer, (rarely a good thing), or alternatively continue reading past the supposed end of data in the source buffer to get access to a copy of data that we might not otherwise have, (because the spurious trailing copy hadn't been subsequently zero'ed out along with the data that was actually previously intended to be in that buffer), and more. The mind boggles.

Looking closely at the garbage characters after the real output is also revealing. We see a lot of à which is 0xC3, and then Ä which is 0xC4. These are bytes that introduce two-byte UTF-8 characters. So when the data is erroneously dumped to the console a second time, it's not even being interpreted as UTF-8, but just dumped directly.

This behaviour can be used to hang the console in a different way to what we saw before when we sent the 0xC4, 0x80 sequence. That basically just swallowed the next byte of input, but we can do better, (or worse, depending on your viewpoint), than this.

We'll test the situation with our local changes to the kernel code backed out. This means that any mis-behaviour we discover would be reproduceable on a stock installation of this release.

The following perl script should repeatedly output the characters at codepoints 160 - 255:

Due to the buggy kernel code, the console hangs after it's output about 2048 characters.

Outputting the 0xC4, 0x80 sequence repeatedly, as the following perl script does, also causes an interesting effect:

When this script is run, the console will appear to hang. Typing any character will cause that character to repeat, and further characters can then be typed which will be concatenated to it and repeatedly sent to the console. At some point the repeating string is cleared and we start again. Eventually it will just hang as before, and we'll need to ^C ourselves out of it.

Although this is fun to see, especially on an unmodified -release kernel, it doesn't seem to be particularly exploitable.

Whilst our changes to the UTF-8 decoding logic in the wsemul_getchar function haven't fixed all of it's shortcomings, it should at least now be internally consistent. In other words, it processes the intended number of bytes, and updates it's own ‘len’ and ‘buf’ values correctly. Certainly whilst testing it with valid UTF-8 sequences, we wouldn't expect to see any problems.

So the buffer mis-management issue must lie elsewhere.

The getchar function is called from wsemul_vt100_output, and just by looking casually at the code we can see an interestingly named variable, ‘processed’, being incremented by exactly one for every character that is sent to wsemul_vt100_output_normal. The value of ‘processed’ is then eventually used as the return value for the function.

Even without studying the code further, it seems likely that this should be counting bytes and not characters.

This theory can be tested in a very quick and simple way by adding two lines after the processed++ statement following the call to wsemul_vt100_output_normal:

Obviously this is far from being the correct fix, but it will allow us to see if we're on the right track.

Testing the characterset program again, we get a much better result.

Now the characters are displayed as expected, and there is no spurious trailing output.

Displaying lots of un-interrupted multi-byte characters will still cause the console to hang, confirming that the testing code we just added to further increment ‘processed’ isn't sufficient to solve the problem.