Smart phone chorded keyboard

In 2007 I blogged about five finger keyboards, also known as chorded keyboards, as an alternative to the dinky and hard-to-use physical keyboards on mobile phones. Five buttons located on the edge of the phone, under the fingertips in normal use, could be used to enter text whenever needed, without having to waste time moving fingers from one key to another. In 2008 I built a simple demo of the idea in JavaScript.

Over the past few years smart phones have become common, and their preferred entry method is via a pop-up "soft" keyboard on a touch sensitive screen. Mostly we use one finger to hunt-and-peck text into these, but some interesting alternatives have been developed, like sliding your finger across the keyboard from key to key, or using gestures. Gizmode has a nice article on some interesting new ways to capture text into an Android phone.

Android 2.3 Soft Keyboard

If we implemented a chording keyboard on a touch-sensitive screen, how would it compare with what's out there already? Let's start with the default soft keyboard that ships with Android 2.3. It has 34 soft keys, of which 29 yield characters and 5 functions (e.g. backspace). You tap a key briefly to get its character or function. 28 of the character-bearing soft keys have a extra character value that you access by pressing and holding the soft key, providing a total of 62 characters and functions. These extra values include the numerals and special characters used for punctuation and maths. As on PC keyboards, the extra values appear above base values on each soft key, so it's easy to work out which soft key to press in order to get an extra character. For example:

3 e

15 of the Android 2.3 keyboard's soft keys carry additional letter variants, that is, the base letter plus a diacritic of óne type or ànother, 57 variants in all. They aren't visible on the soft keyboard, but each relates to a base letter on the soft keyboard. For example, you access the diacritic variants of the letter "e" by pressing and holding the base letter "e". A pop-up appears with its diacritic variants following the default "3" which appears in the upper half of the soft "e" key:

3èéêëęě

You choose between these options by sliding your finger sideways. The focus moves across the list of options in sympathy. The character that has focus when you lift your finger from the screen is taken.

Soft Chording Keyboard

Let's design a phone app to capture text using chording key principles, but with the fingers touching soft keys on the phone's screen. Ideally, the fingers should only have to move up and down in one place. With my adult male fingers, I can only fit three fingertips across the width of my HTC Wildfire's touch screen. A classic chording keyboard with three buttons would give us 2^3-1 = 7 different characters, way too few (I'm using the notation A^B to mean A to the power of B). If we take into account the sequence of touches and allow for sequences of 1, 2 or 3 touches then we get 3*2*1 combinations that include all three fingers plus 3*2 combinations that include only two fingers plus 3 combinations that use only one finger = 15 different characters, still way too few. If we take into account the sequence in which the finger tips touch the screen and also the sequence in which they leave the screen then we can encode 3*2*1 * 3*2*1 + 3*2*2 + 3*1 = 51 combinations. Better, but we're still short of the 62 characters that the Android 2.3 keyboard can encode, ignoring case shift and diacritics. We will have to allow individual fingers to press and release their assigned soft key more than once in a coding sequence. Then the sky is the limit.

While we're about it, let's note that most pop-up keyboards take up a lot of real estate. If the user needs to fill in a multiple field form then the pop-up keyboard will likely cover most of the form. It would be nice if our chording keyboard used up less screen space, but still guide the novice user through character selection. Here's a sample "home page" for our pop-up chording keyboard:

Backspace	Blank	Shift: Aa	New line
Put three fingertips on this line to enter text

We have a list of four common functions/characters across the top. A production version would use symbols rather than words. The lower half is where the user's fingertips will rest. We could predefine three blocks and require the user to put a fingertip in each, but lining up fingertips with blocks would take time. Instead, the user is free to place three fingertips anywhere on the line of text, and the keyboard app will find them. When this is done, the top line is replaced by cues that lead the user through the process of selecting a character. The user then lifts and lowers fingers in various sequences to choose characters. At any time the user can lift all fingers simultaneously to cancel the process and return to the home page. Once selection starts, here's how the first set of cues might look:

a-m	n-z	12#
(==)	(==)	(==)

a-m gives access to the first half of the alphabet, n-z to the second. 12# gives access to numbers and special symbols. (==) marks where the user's fingertip happens to lie. The app writes the cues above each fingertip. If the user lifts finger 1, selection of a character in the range a-m will start. The cues will change, and the keyboard pop-up might then look like this:

a-d	e-h	i-m
(---)	(==)	(==)

(---) marks where fingertip 1 was before it was lifted; the other two are still down. The keyboard app wouldn't actually display the (==) or (---), they're only there to assist with the explanation. The first cue a-d is italicised to indicate that the user should lower that finger to choose that option. If the user lifts finger 2 then the pop-up might change to:

e	f-g	h
(---)	(---)	(==)

Bold cues show tell the user that if that soft key is chosen, the selection process will arrive at a character. In this case, lowering finger 1 will select the letter e. After that, the user could lower finger 2 so all three fingertips are on the glass. The keyboard would return to the top character selection page, ready to start the next character.

Accessing Diacritic Variants

We can allow users to access diacritic variants of letters in a similar way to the Android soft keyboard. Once you complete a sequence that yields the letter "e" for example, then the selected letter, and the secondary character that shares its soft key, and the diacritic variants of the selected letter could appear in the cue area like this:

e3èéêëęě
(==)	(---)	(==)

The selected character "e" has the focus. If you ignore this cue and lower finger 2 to start selecting the next character, "e" will be chosen as the next input character. If you pause, focus will move to "3", the secondary character that shares the "e" soft key. If you pause longer, focus will move cyclically further to the right, wrapping back to the left once it runs off the right edge, so you can select any variant just by waiting. Alternatively, you can slide your fingers to the left or right, and the focus will move left or right in sympathy. Whatever character has focus when you lower finger 2 will be selected. Fingertips that slide off the side of the screen will be ignored. The base letter and the focus will be located at the right of the string of options if fingertip 3 is the only one in contact with the screen, so that it has room to slide left. After a sideways slide you might have to lift all fingertips and re-centre them, which is undesirable, but you won't have to be too fussy about where you locate them, so it should be quick.

Character Code Points

Starting from the home position with three fingertips on the screen, you have a choice of three movements – raise any one of your three fingers. After any of these movements, you have another three choices, and so on. You are navigating a trinary tree, with a single character at each leaf position and cues at non-terminal nodes. A five level balanced trinary tree would have 1 root with 3 child nodes, 9 grandchild nodes, 27 great-grandchild nodes and 81 great-great-grandchild nodes. Each successive level is reached by raising or lowering one fingertip. Of the 27 great-grandchild nodes, one will have all fingers raised, which position we reserve to end the current character selection sequence as an "Oops!" gesture. So we will lose three potential code points, leaving 78. Of these 78 leaf nodes, some have all three fingers down. We discard these because that configuration is required to start the next character. All fingers down can be reached after four moves by raising two fingers and then lowering two fingers, 3*2*2*1 possible sequences = 12 code points; or by raising and lowering a finger and then doing the same again, 3*1*3*1 = 9 code points (we accept "all fingers down" as valid if this happens at level 2). That leaves 57 code points, five less than the number of characters and functions that appear on the Android soft keyboard. But one of the Android keys is not needed (12# selects special characters), and our home page includes four common functions to compensate. All of the code points can be accessed with only four finger up-down movements.

In practice, we may choose to follow the example set by Samuel Morse and allocated shorter code sequences to more common characters, and longer code sequences to less common sequences, to reduce the average number of finger movements needed to code a character. The sample screens above allocate 2/3 of the code space to the alphas, although they form only 42% of the Android character set. We also need to group the special characters so that we can give cues that will guide the novice user to find each character required. They can be divided into the 10 numerals, characters that can be drawn with one continuous line stroke @&()_-,./ and characters that require more than one line stroke !#$%+=:;"?

If we introduce longer and shorter code sequences then we will also be able to include other characters that are present on the PC keyboard but not the Android 2.3 keyboard, such as `~[]{}<>

How Quick Would It Be?

Could a chording keyboard achieve reasonable text entry rates in the hands of a practised use? You can benchmark your own digital dexterity by resting your wrist on a flat surface and drumming three fingertips on it, as if you were impatiently waiting for someone. Try different patterns. It's an easy action for the fingers. Many folk can clock 10 taps per second; that's 20 finger movements per second. If we use a balanced trinary tree for our coding and include an extra movement to get all fingers on the screen to start each character, we need 5 finger movements per character entered (ignoring capitals and diacritics), so our fingers are theoretically capable of entering 20/5 = 4 characters per second or 48 words per minute, faster than the average typer on a full-sized PC keyboard. How fast we would be when typing a given text is another question which only experimentation can answer. Of course you can type letters at a furious pace if you simply drum your fingertips on a PC keyboard, but you will get the same three or four letters repeated over and over again. Once you start moving your fingers around the keyboard to find particular letters, the process slows down greatly.