The topic of contextual information in user interface and the lack thereof in screen reader interfaces creates a lot of controversy and discussion among people who think about such things. A few years ago, ATIA held an AT/IT compatibility meeting which kicked off its AT/IT compatibility committee as well as its AT/AT compatibility committee. The membership of the AT/IT group contained people from mainstream companies (Microsoft, IBM, Adobe, etc.) and AT companies (FS, AI^2, Medentech and others). Its agenda included helping define new methods of communicating between AT products and mainstream software packages in order to improve the user experience.
Well, a few years have passed, lots of new versions of JAWS, Window-Eyes and HAL have shipped. A new version of MS Office (2003) went out the door and these releases demonstrate little progress in bringing the screen reader user more contextual information. Apple has released its screen reader and Microsoft’s next OS release will contain a new API called “UI Automation” which claims to hold enough information to move the art forward. Through a lot of terrific work by our friends at Sun Microsystems and IBM, a new AT/IT compatibility protocol was published by the AT/IT committee that looks a lot like the gnome accessibility API with some extensions and, to my knowledge, hasn’t been used in any product anywhere.
Other than direct “screen scraping” (taking information written to the screen and delivering it verbatim to the user) the two most commonly used techniques for getting data out of an application and delivering it to a blind user have been MSAA and Microsoft’s Object Model interface. As a Microsoft manager once said, “an MSAA object is like an amoeba, it knows a bit about itself and its purpose but nothing about its surroundings,” which makes the context problem impossible to solve using MSAA as the soul source of information. The MS Object Model provides much more information about context but remains so limited that a screen reader cannot do much to provide its users with information rich enough so they can perform at an even level with their sighted counterparts.
What do I mean by contextual information?
When a sighted person views a spreadsheet, they can focus their attention on a specific cell but, by seeing the surrounding cells, they can infer certain pieces of information about the cell they care most about. A screen reader user with JAWS can learn a lot about the cell and its context by using some of the more advanced JAWS features but the limits upon them slow their data gathering much more so than a person who, by diverting their gaze, can learn a lot about a single data point. The reason for using JAWS as the example is because it leads the pack in delivering contextual information in important applications, I will criticize it a bit in this article only by suggesting that more can be done in the future. Window-Eyes and Freedom Box System Access have both done some catching up in this area in word processors and should be encouraged to continue to innovate as well.
Screen readers present information in a one dimensional manner. They either deliver a long string of syllables through a speech synthesizer or deliver a few characters on a single line of a refreshable Braille display. Multi-line Braille displays are cost prohibitive and have made few inroads into the market. The ViewPlus tactile display, while expensive, does an amazing job of delivering contextual information and, for users who can gain access to such products, it may be the feel of the future.
How can screen readers improve?
Three dimensional audio is now cheap and ubiquitous. On Tuesday, I paid $99 for a Creative Labs audio card for my laptop that supports all of the latest Direct X features, Dolby 5.1 & 7.1 as well as a few other goodies. So, as I have mentioned before, the screen reader vendors should start researching two and three dimensional interfaces in audio and, using tactile displays like the one from ViewPlus, two dimensional touch interfaces.
Such interfaces will make using products we blinks already have, like word processors and spreadsheets more efficient and will open up accessibility to very graphical programs like Microsoft Visio and VisualStudio.
Why is this important?
Because we blinks need to compete in the workplace. We desire career advancements and hope to move up in organizations. Today, it is impossible to provide “reasonable accommodations” to programs from which screen readers cannot gather enough information to make them usable. The technology is available in Direct X, in the tactile displays from ViewPlus and in Microsoft’s Object Models. Now, we need the AT companies to be creative and discover ways to deliver it to us. JAWS has set a pretty high bar with the information it can expose in Word, Excel, PowerPoint and MS Project but still has a way to go. The other players need to catch up to JAWS and then start considering adding more dimensions to the reality of screen reader users like me. This is one of my real hot button topics so expect to read more about contextual information and multi-dimensional interfaces at Blind Confidential in the future.
Now, back to the ICADI conference to learn more about smart homes and how we blinks can use them to improve our standard of living. Thus far, this conference has been very cool.