RUBBER mallets began appearing in offices soon after Microsoft launched its Windows 95 operating system. An enterprising businessman realised that, faced with inexplicable program glitches and unfathomable menus, frustrated users needed to vent their anger on their recalcitrant computers, but without doing them too much harm. Although computers are championed as tireless servants, when it comes to using them it is the user who has to adapt to the computer, not vice versa. As computing plays an increasing part in people's lives, much research is being focused on making computers genuinely friendlier and more useful.
This is why “sentient computing” has begun to capture people's attention. By adding sensors to today's computing and communications technology, sentient computing seeks to take account of a machine's environment in order to make it more responsive and useful. Sentient computing systems are always on, ubiquitously available, and can adapt to their users. In short, they seek to become real help-mates. To quote a European Commission report, the aim is to create “convivial technologies that are easy to live with”.
According to Emile Aarts at Philips Research in Eindhoven, the Netherlands, these convivial technologies will emerge in a number of ways. User interfaces, for example, will move from “cognitive” to “intuitive”.
So, instead of having to turn the television on, the TV will know what you want by combining an understanding of what you say, your expression, your gestures and even how you walk. Ideally, the television set will also be aware of the context—say, turning the sound down if you are on the telephone. And it can recognise you personally, remember what your favourite channels are, and which programmes to record. All this will happen in an environment where computing and communications are as invisibly available as electricity or water.
Some products already capture the spirit of sentient computing. When the makers of mobile phones first put the mouthpiece on a flip-up cover, users had to open the handset and irritatingly press a button to answer a call. It did not take long for manufacturers to add a sensor so that opening the flip-up cover answered a call and closing the cover ended it. A small change, for sure, but one that made the flip-phone far easier to use.
Similarly, Microsoft researchers have given wireless PDAs (personal digital assistants) new user interfaces by adding tilt sensors and accelerometers. Users can scroll through documents by tipping the PDA back and forth—as if they were controlling the ball-bearing in a toy maze. Lifting the PDA to the ear creates sensor outputs that cause the device to make a phone call. Researchers have even used the accelerometers to recognise walking patterns so that the PDA can decide whether to accept or divert a phone call.
Staking out the territory
With such usefulness in mind, research on sentient computing has become increasingly active in information technology (IT) laboratories in Europe and America. Projects under way at the University of Cambridge, the Massachusetts Institute of Technology (MIT), Philips and elsewhere are attempting to stake out the territory by delving into such topics as “ambient intelligence”, “ubiquitous computing”, “aware environments” and the “intelligent home”. In fact, groups such as Andy Hopper's at Cambridge have been working on sentient computing since the late 1980s. But the swell of interest caused by developments in mobile communications and entertainment has attracted lots of others to the field.
MIT, for instance, has its Project Oxygen, a broad research effort on pervasive human-centred computing sponsored by Acer, Delta Electronics, Hewlett-Packard, Nippon Telegraph and Telephone, Nokia and Philips. According to Victor Zue of MIT's Laboratory for Computer Science, the Oxygen project sprang from curiosity about four issues. The first was to get computers to help people achieve more by doing less. The second was to understand what happens when computing and communications become basically free. The third was to return computers to the service of users. And the fourth was the desire to make computers understand people's needs.
Not a bad start—if modern households are anything to go by. Affluent homes today are flooded with technology—from DVD players and widescreen TV sets to multiple computers, home networks, wireless devices, and a plethora of fixed and portable music players. They are also swamped with information, streaming through phone lines and broadband internet connections, analogue and digital television and radio, and recorded media such as CDs and DVDs made at home or brought in from record or video stores. The pile of remote control units makes a clear statement that there must be a better way of telling machines what you want.
Another reason why sentient computing is becoming popular is that the technology is at last getting good enough to make it all happen—if not now, then at least with computing, communications and storage technologies that are on the near-term horizon. Also, the infrastructure needed to make sentient computing a reality—communications that are ubiquitous and always connected—is getting established as mobile and other wireless networks spread.
Finally, there is the profit motive, or lack thereof. With the IT industry having been in the doldrums for two years now, firms are desperate for something new—anything—to kick-start demand.
At Philips Research, Mr Aarts believes that the availability of technology and infrastructure, plus people's desire for simpler and friendlier user interfaces, is what lies behind the surge in interest in sentient computing. But beyond such factors lies a significant change in the way business is done—a shift to an “experience-based economy”.
Like it or not, the future is Big Brother—so let's talk about it.
This is the next stage after moving from commodities to manufactured goods and then to services. When services become commoditised, so the argument goes, the way to re-infuse them with economic value is to present them as experiences—hence the squashy sofas that Starbucks uses to differentiate itself from ordinary coffee-shops. With mass-customisation, says Mr Aarts, the key differentiator for future products will be the experience they give you. Get that right and market success will follow close behind.
A number of issues need to be looked at, however, if sentient computing is to be brought into everyday life. Wireless networks will have to merge so that users can roam seamlessly between Bluetooth and WiFi connections at home and mobile networks when travelling. Accurate location information will be important, not only to pinpoint users in the landscape, but also to position them in rooms. Sensors will have to become ubiquitous in products, as well as the environment, to provide the vital contextual information. And the systems driving sentient applications will have to be able to integrate massive amounts of widely distributed data, so they can be presented to each user in a personalised way.
Programming with space
Once these issues are overcome, new opportunities will emerge for producing human-centred IT. Imagine being able to “program with space”, using information about where you are in your home to control the lighting, ensure that only your nearest telephone rings, or that the television programme you are watching follows you from screen to screen as you move between rooms. Imagine also that the display screen and the computing functions that drive it become separate entities rather than part of the same thing. The movie you were watching on the big screen at home could then continue to play automatically on your mobile handset when you left the house.
The mobile world is getting its share of attention. Both MIT's Oxygen project and a team at the Interuniversitair Micro-Elektronica Centrum (IMEC) in Leuven, Belgium, believe that developing a kind of “Swiss army knife” of mobile electronics is the key to providing sentient computing on the move. At MIT, the all-purpose tool, called Handy 21, combines a PDA with a camera, wireless network, a microphone, speaker and accelerometer.
Meanwhile, the IMEC group has launched its “multi-mode multi-media” (M4) program as a way of delivering any bit of information, anywhere, at anytime. Rudy Lauwereins at IMEC believes that the basic technologies to do this are already available, but have yet to be integrated into a seamless package.
The Belgian group has proposed a flexible information appliance that takes many of the tasks now performed by specialised pieces of equipment (eg, mobile telephony and music playback) and recasts them as software applications running on a powerful, adaptable computing engine. According to Mr Lauwereins, it will be 2006 before the chipmaking processes improve enough to give the device a reasonable battery life.
Once these devices are available, how will they affect people's lives? The European Commission report on ambient intelligence describes a fanciful scenario in which Maria, a businesswoman, arrives at a foreign airport and walks through immigration unchecked, thanks to a wristwatch computer that presents her ID and visa for validation. At the kerbside, a rental car unlocks itself as she approaches and then guides her to a reserved parking bay at her hotel. As Maria enters her room, it adapts to her preferences by adjusting the temperature, lighting and choice of TV and music channels. Later, she talks to her daughter on a video wall while inserting local references into her presentation for the morning using the hotel's special software.
Yes, all science fiction today. Along with hardware and infrastructure problems, researchers still need to work out how to scale up the limited sentient computing systems they are now tinkering with. They also need to create co-operative file systems that work globally, and they need to make it easy for sentient applications to find network printers and screens that are close to users. Beyond that lie touchy issues of security, privacy and permission.
Is privacy something we will trade off for convenience? Sentient computing, with its reliance on knowing where users are, could certainly make the loss of privacy more serious. Dr Hopper at Cambridge University advocates acceptance and debate. Like it or not, he says, “the future is Big Brother—so let's talk about it.” But Dr Hopper has been wearing a location tracking device for the past 12 years—and has learned to live with the loss of privacy and to appreciate its benefits.
Instead of users being tracked constantly, how about using “permission-based” sentient applications, where users have to opt in if they want the benefits? Purists such as Dr Hopper argue that this breaks the sentient computing paradigm. But Dr Zue at MIT likens permission-based systems to having a telephone in the bedroom. We opt to have it because it delivers a useful service, despite the fact that it could also be used to monitor us while we sleep. Perhaps, ultimately, sentient computing will show that privacy is not all it is cracked up to be.
This article appeared in the Technology Quarterly section of the print edition under the headline "The sentient office is coming"