Linux Accessibility For Disabled Users As Linux matures and proliferates, the disabled community is asking how they too can access this important operating system. We have a legal and moral obligation to make Linux accessible to as many individuals as possible. Disabilities may include blindness (can't read the output), and motor impaired (can't type the input), to name just two. Given the difficulty of programming in the kernel, and maintaining patches while versions of Linux march along, we are not surprised to find an assortment of adapted applications. It is relatively easy to make a user program send its output to a synthesizer, or read from an alternate peripheral (rather than the keyboard). EmacsSpeak for the blind is a good example. However, we must remember that a program is being adapted, not the computer. Often the program is extremely powerful, such as emacs or bash, but it is a program nonetheless. There will always be some users who would rather adapt the entire computer, as has been done for Dos and Windows. Fortunately this task has become much easier, thanks to a standard interface that supports adaptive modules. When you run `make config', say yes to "accessibility adapters", and the resulting kernel will support adaptive modules. By definition, an adaptive module redirects or restructures keyboard input or console output in a manner that compensates for a particular disability. These streams were not available to loadable modules in earlier versions of Linux, but they are now. Here are some sample adapters. Blind: send console output to a speech synthesizer. Function keys allow the user to review the contents of screen memory. Blind-deaf: send console output to a braille device. One hand: Keys are remapped to minimize cross-keyboard movements. This may include key chords, which cannot be implemented via loadkeys. No hands: A sip and puff peripheral sends a limited set of scan codes to the keyboard input port. An adapter reads them in sequence and translates them into characters. This may include word completion/prediction algorithms and user defined macros. The accessibility interface is defined in linux/accessibility.h. Module developers should include this header file, along with module.h and any other files needed for consistency. After you've read this document, please review accessibility.h. The adaptive module defines a set of operations, actually functions, that the kernel invokes. These are similar to file operations or inode operations, but they are driven by console output and keyboard input. These "callback" functions are defined below. void (*tty_start)(int minor); void (*tty_stop)(int minor); These functions are called when a tty is allocated or freed, respectively. A tty is allocated when it is opened for the first time. This action invokes tty_start(), passing the minor device number of the newly created tty. Subsequent processes may open and close this tty, but that will not trigger any adapter operations, until the last process closes the tty. That frees the tty and invokes tty_stop(). Some adapters use tty_start() to allocate a structure for each open tty. This can be thought of as an extention to struct tty_struct, but it isn't defined in linux/tty.h; it is defined by the adapter. Tty_stop() can then be used to free the structure and stop any reading that might be in progress on that tty. Developers may want to think carefully before deallocating resources inside tty_stop(). Under Redhat, the first tty driver, tty1, is opened to run rc.sysinit, then closed. Then all the virtual terminals are opened as you enter a higher run level. If you deallocate resources when tty1 closes, there will be no log of the output of rc_sysinit. This is probably not what you want. My adapter allocates its buffers through tty_start(), but basically ignores tty_stop(). It then deallocates everything it knows about in cleanup_module(). int (*tty_out)(int minor, unsigned char c, int isEcho); This function is called whenever an output character is generated. The first parameter holds the minor device number, which is between 1 and 63. The second parameter is the output character. It's high order bit may be significant. Finally, the third parameter determines whether this character is an echo of an input character, according to the line discipline of the tty. I use this information to generate a unique sound when the user is typing capital letters, thus the user will not type a paragraph in caps lock by mistake. You wouldn't want these tones to appear whenever the computer prints capital letters, so I only generate the tones when isecho is nonzero. The return value is a bit map as follows. 01send the character on to the screen, or virtual screen (virtual terminals are fully supported). If the character is eaten as part of a recognized escape sequence, this function should return 0. 02Send an escape to the screen before sending this character. This is used when we thought we were running a talking escape sequence, but then we didn't get the right follow-on character, so we want the tty to send the escape that was swallowed earlier. 04Take a realtime break. It took a while to process this particular character, for whatever reason, and we don't want to monopolize the CPU. My system sets this bit after it generates the sound associated with the return character. The sound is made using CPU cycles -- quite a few of them -- so it is best to let another process have a turn. int (*keystroke)(char *key_p, int *shiftstate_p, char upflag) ; The key code and shift state are passed into this routine. Keycodes are fairly (though not entirely) standard representations of keys on the keyboard, and the bits of shiftstate, indicating shift, control, alt, etc, are documented in the Linux manual. See dumpkeys(1), showkey(1), and loadkeys(1). Upflag is set if the key is being released. Most adapters simply return if this flag is set. they are only interested in key press events. But if you want to recognize key chords that activate specialized functions, this interface will support that. Note that key and state are passed by reference. The adapter can change the key and shiftstate, and Linux will process the new key and state, as though that were entered at the keyboard. This is rarely done however. In fact you have to be very careful with this. If you change x to y with upflag off, you'll want to do the same with upflag on, press and release, so that Linux can maintain its bookkeeping. As I say, most adapters don't change the key codes. But they sometimes eat the keystroke, as described below. The return value is boolean. If zero is returned, the key has been eaten by the adapter. For example, the keystroke might cause the module to begin reading. The running process doesn't need to see it. Conversely, a nonzero return routes the key to its destination. This is usually a process reading from /dev/console, but it could be a meta function such as changing virtual terminals or the famous control-alt-delete reboot. WARNING!! Unlike the previous functions, this one is invoked as part of an interrupt routine. Specifically, it is called from the bottom half of the keyboard interrupt. Hardware interrupts are not disabled, so you won't lose data from a serial port or ethernet connection, but some Linux functions are suspended while inside this software interrupt. You should not hold the CPU for too long. When a speech function cannot be executed right away, because the synthesizer is busy (for instance), put it on a pending queue and deal with it later. Because this routine is triggered by a keystroke, it is entirely asynchronous with respect to the kernel. As a result there are certain operations you cannot perform, such as allocating memory or other resources. The tty_start() routine is a better place to allocate all the resources you will need to perform your functions. void scroll(int minor, int t, int b, int nr) ; void cursorloc(int minor, int x, int y) ; When a console screen, or section thereof, scrolls up or down, the kernel calls scroll(), passing the top and bottom of the region, and the number of rows that the region has scrolled up. A negative number means the region has scrolled down. If the adapter is reading from screen memory, it may want to move its reading cursor accordingly, to keep in step with the moving text. Whenever the cursor moves, the kernel calls cursorloc(), passing the new x and y coordinates. In a typical sequence, a program generates output, the tty driver passes the output character to the adapter, the adapter returns 1, the tty driver sends the character on to the console, the console prints it, and moves the cursor one to the right, or down to the next line if we wrap, then sends the new cursor location to the adapter via cursorloc(). void (*polling)(void); Most adapters employ an asynchronous thread to implement continuous reading. This thread monitors the speech synthesizer and transmits text while Linux executes other processes. Thus the blind user can sit back and listen to a screen full of information while somebody else computes the first million digits of pi. For simplicity, this thread is usually implemented by a polling function. In other words, a function wakes up ten times a second, checks the status of the synthesizer, and if the unit is not busy, sends it more words to speak. This is not the most efficient design, but we don't really need to mess with interrupts for events that only come two or three times a second. The adapter supplies the polling function and the period. extern int register_accessibility_device(struct accessibility_device *dev); extern void unregister_accessibility_device(struct accessibility_device *dev); The adaptive module calls a register function to attach itself to the kernel, and an unregister function to detach itself from the kernel. This is usually done from init_module() and cleanup_module() respectively. The function pointers are passed to register() and unregister() inside a structure that describes the adapter. See accessibility.h for more details. Register() could fail if another adaptive module is currently loaded. Otherwise it sets the function pointers as you indicate and calls tty_start() for each open tty. Similarly, unregister() calls tty_stop() for each open tty, and restores the kernel to its original behavior. Obviously the open ttys aren't going away, but the adapter is, and this seems like a simple way to clean things up. The adaptive module, like any other module, is limited to the symbols that have been exported by the kernel. This list is growing all the time, and most of the symbols needed by a speech adapter should be availble. These include functions to access the serial port, for speech synthesizers on ttyS0 through ttyS3. However, if you are writing an unusual adapter, you might find you need a function or variable that has not been exported. In this case you may indeed need to patch the kernel, but it is a simple patch, and you should have no trouble incorporating it into the next release of Linux. In addition to exporting preexisting symbols, I have added some functions to the base kernel -- functions that most adaptive modules will need -- functions that developers don't want to reinvent and maintain over and over again. Future releases will include more functions that are common to many adapters. extern void kd_mknotes(const short *notes); An array of notes is pushed onto an internal sound fifo. These notes are then played by an asynchronous thread, using the PC speaker. If the fifo is full, the incoming tones are discarded. Thus if a program issues 100 note strings in rapid sequence the first dozen fill the fifo and the rest are silently lost. The fifo then drains over the next few seconds. Of course this isn't likely to happen in practice. Normally we place beeps and tones into a near-empty fifo, whence they are played immediately. If the computer has no toggle speaker, this function is a stub. Future versions will send sin waves to the sound card. The parameter *notes is an array of shorts. Each note consists of two shorts: frequency and duration. A frequency of -1 is a rest. A frequency of zero ends the list of notes. Duration is measured in hundredths of a second, and frequency is given in hurtz. extern int vc_screenParams(int minor, short *nlines, short *ncols, short *x, short *y); extern int vc_screenimage(int minor, short *dest, int destsize); extern int vc_screenItem(int minor, int x, int y); The first function retrieves parameters for the designated console, returning -ENODEV if the minor number is not associated with an active tty. The adapter may wish to allocate a buffer of shorts, nlines*ncols in size, then use the second function to retrieve a copy of screen memory, frozen in time. Alternatively, the adapter can use the third function to read individual characters off the screen. Adapters can make use of the /proc file system to pass information back to the user, or on to the synthesizer. This is generally more flexible than new ioctl directives or system calls. The directory /proc/accessibility is intended to house various adapters. Create a subdirectory for your adapter, then add more files under this subdirectory as you see fit. For example, my Jupiter adapter makes the text buffers available under /proc/accessibility/jupiter, so your ineractive session can be saved to a file at any time. The session associated with tty3 is captured in /proc/accessibility/jupiter/buf3. To prevent others from evesdropping on your work, this file is owned by root, or by you, if you pass your uid as a module parameter. This is merely an example; each adapter can add its own files under /proc/accessibility/xxx, for various purposes. Below is a sample adapter. It prints messages when it attaches and detaches itself, intersepts shift F5, and swallows escape q. ---------------------------------------------------------------------- /********************************************************************* adapter_generic.c: skeleton for an adaptive module. Compile this using the C flags that correspond to kernel modules on your system. The following flags will probably work. CFLAGS = -D__KERNEL__ -I/usr/src/linux/include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -pipe -fno-strength-reduce -m386 -DCPU=386 -DMODULE Then run insmod on the resulting object file. insmod adapter_generic.o period=7 Run rmmod when you want to remove the module. *********************************************************************/ #include #include #include int period; MODULE_PARM(period, "i"); /* i means integer */ /* Make a little beep every period seconds, or not, if period = 0. */ static void pollFunction(struct accessibility_device *dev) { static const short beep[] = { 3000,3,0,0 }; kd_mknotes(beep); } static void tty_start(struct accessibility_device *dev, int minor) { printk("tty start %d\n", minor); } static void tty_stop(struct accessibility_device *dev, int minor) { printk("tty stop %d\n", minor); } /* Eat escape q, as though it invoked a particular function */ static int tty_out(struct accessibility_device *dev, int minor, unsigned char c, int isecho) { static int escstate = 0; if(escstate) { escstate = 0; if(c == 'q') { /* perform the function */ printk("escape q function\n"); /* Return 4 if it took a while to run this function */ return 4; } return 3; /* print escape and the current character */ } if(c == '\33') { escstate = 1; return 0; } return 1; } /* Let shift F5 perform a special function */ /* Let alt m be a macro */ static int keystroke(struct accessibility_device *dev, int key, int shiftstate) { if(key == 0x3f && shiftstate == 1) { /* perform the function */ printk("f5 function\n"); return 0; } if(key == 0x32 && shiftstate == 2) { puts_queue("macro"); return 0; } return 1; } static struct accessibility_operations myops = { tty_start, tty_stop, tty_out, keystroke, NULL, /* scroll function */ NULL, /* cursor location function */ NULL, /* might become pollFunction */ 0, }; static struct accessibility_device mydev = { ops: &myops, }; int init_module(void) { if(period) { myops.polling = pollFunction; myops.period = period*100; } return register_accessibility_device(&mydev); } void cleanup_module(void) { unregister_accessibility_device(&mydev); } ---------------------------------------------------------------------- Written and maintained by Karl Dahlke. karl@eklhad.net