Many of the companies presenting at this year's NVISION are discussing human-computer interfaces (HCI) in one form or another. Any discussion of
"visual" computing must ultimately address how the presentation of content in space (and the mode in which it's presented) impacts how we
perceive and interact with it. The mouse+keyboard approached has served us well for the past 40 years or so, as have game controllers (albeit
for not quite so long), but Nintendo's Wii has conclusively demonstrated that consumers are interested in moving past traditional control
Actually developing those control systems, however, is a difficult, complex, process. Many of the hardware components that are needed are still in their infancy (or haven't been invented yet), and programmers will need time to find the best ways to take advantage of what the hardware can actually do. Despite these difficulties, interface technology is a hot topic for NVIDIA's first expo—though ironically, one of the most interesting interface projects I'm aware of isn't on the table.
Researchers have been attempting to improve the tools available to quadriplegics for decades, but with limited success. Sip/puff systems are available, and relatively inexpensive, but are limited to just four commands, while the sophisticated eye-tracking sensors available on the market are often slow, prone to error, and extremely expensive. Other options, such as equipment that can translate head movement into mouse movement are available, but users of such devices tend to tire quickly. Research in all of these areas is still ongoing, but there's a new approach in development.
Dr. Maysam Ghovanloo is an assistant professor at the Georgia Institute of Technology, and head of that school's GT-Bionics Lab. He's working on a control system that could give quadriplegics a much wider range of movement than they currently enjoy, by using their tongue. Since the tongue connects directly to the brain, it's not effected by neck injury, and typically remains fully operational, even in patients that have lost all ability to move.
Prototype systems have already been built—a video of one in action is available here—and it functions through use of a virtual keyboard. A permanent magnet is attached to the tongue, with a control unit underneath and various sensors attached to the teeth, bracer-fashion. These sensors detect movement of the tongue inside the mouth, and transmit this data wirelessly to a control module, which then reacts appropriately, depending on what the wearer has signaled. The prototype unit pictured here was built from off-the-shelf technology and is obviously a few years old—there's no word yet on whether newer, more sophisticated designs are currently in the works.
Currently, the device requires one to wear a rather bulky set of headgear, and the mouthpiece on the prototype has been described by at least one graduate student familiar with the project as "grotesque." Currently, the device is limited to six commands, but there are plans to introduce more sophisticated sensors capable of tracking a higher number of movements as well as determining subtle differences between movements. The research team hopes to create a commercially viable product with a price point between that of the suck-n-blow systems at one end, and the expensive eye-tracking systems at the other.
The vast majority of us might have precious little interest in a tongue-driven interface, but this type of research indirectly benefits from the hardware and software being developed commercially. I wouldn't go so far as to say that the mouse and keyboard are going to go away, but signs point to a not-so-distant future in which we interact with a computer in ways significantly different than what are available now. Adopting and adjusting to new interface technologies will undoubtedly take awhile, but for certain people, this type of sea change could reopen their ability to explore the world in ways that are currently impossible.
- Dr. Maysam Ghovanloo: "Tongue Drive Assisstive Technology"