Cover Story
Microsoft: The Speed of Thought
Photos by Brett Patterson/Abrivo Studio

Andy Wilsonís office at software giant Microsoft almost thinks by itself. Wired in among the mayhem of sagging bookshelves brimming with titles like "Neural Information Processing Systems," precision tools, volt meters and computers, are cameras that give Wilsonís office eyes. Two are mounted on the wall near the ceiling, and a pair stare back at Wilson from his triple flat-screen computer monitor array.

Wilson holds a Ph.D. from the Massachusetts Institute of Technology and its famed Media Lab, where he first began to explore his work in gesture recognition. As part of the Microsoft Research department, heís been teaching his computer to recognize the world outside of its neuro-digital central processing unit.


Because Wilson believes that computers can understand the real world in real time. And that we can use them to help us control and organize things in ways we could never imagine.

Facing his monitors, Wilson lifts his right hand. Onscreen we see Wilsonís image, with his hand quickly identified by an illuminated square. The computer is "seeing" him. He points and manipulates objects onscreen. Itís a hands-off interface that may one day eliminate the need for a conventional keyboard or mouse hard-wired to your computer. Think Tom Cruise in Minority Report. That day is just around the corner, and its dawn seems to be coming from Andy Wilsonís office.

Wilson then picks up a long glowing Plexiglas tube filled with microchips and circuitry called the XWand. He aims it at his monitors. It behaves as the cursor, interacting with his computer just as a mouse would. But he slowly aims the device away from the screen, taking it out of the plane of the computer screen, and it becomes a light beam. He points it at a series of lamps on his desk. They light up and dim as he rolls and manipulates the wand. He guides it toward a stereo and music begins to play.

The possibilities, he says, are endless. One day soon, the XWand might easily be the size of a watch or a keychain. Our computers will know us from Adam, and weíll have total personal control over the thousands of embedded microchips and computers in our homes, offices and cars that drive everything from our personal PCs to our home entertainment centers, lighting and heating systems.

"You canít possibly sit in front of every computer you have in your house right now. Whatís it going to be like when thereĎs 10 times as much computation in your house, in your car, in your hotel room?"

Itís all in a dayís work for Wilson. Heís part of a Microsoft brain trust thatís helping to drive the future of computing, and heís having a good time doing it.

Andy Wilson is one of a select cadre of computer researchers in the Adaptive Systems & Interaction group (ASI) within Microsoftís Research group. ASI spends its days figuring out future waves of computing. They live and breathe the rarified air of automated reasoning, adaptation and human-computer interaction. Theyíre on a relentless hunt for new functionalities and computer flexibility, searching for easier ways to interface with your machine.

For those of us who still over-click, freeze up and howl at our infernal machines, these ideas might seem like science fiction. But computers have penetrated so many facets of living, providing light-speed operation of so many tasks, that we can now think of them as little brains that can be harnessed to perform evermore complex and satisfying duties. Scary, but true.

Microsoft co-founder, CEO and chief software architect Bill Gates has long envisioned the computer as an intelligent assistant able to listen, talk, see and learn. In 1991 he began assembling his Research group, peopled with computer industry legends, making Microsoft one of the first software companies to create its own computer science research organization. While many companies have slashed their research and development budgets, Gates has pumped his up. Microsoft spent $6.8 billion (or roughly 20 percent of total revenues) in the last fiscal year on the Research group, which now numbers some 700 on five campuses around the globe. The companyís Web site lists more than 150 current projects, and that doesnít count the top-secret stuff.

The fruits of these brainiacsí efforts are weaving their way into many of Microsoftís software products and services, some spectacular, others more intuitive and in the background. To be honest, we ainít seen nothiní yet.

COMMUNITY AND LANGUAGE Elsewhere around the building other projects are coming to life. Lili Cheng and her Social Computing group explore the ways we use computers to develop community. Cheng, a licensed architect, led a test project called HutchWorld, an experiment in grand interfacing that allowed kids under treatment at Seattleís Fred Hutchison Cancer Research Center to build an online community.

"Itís easy to sit around a lunch counter and have a great idea and say, ĎLetís do it.í The cool thing is that we have enough people that itís a deep bench. Thereís a lot of variety."

That led to other computer-community research that includes KidTalk, which helped children with autism, and Wallop, a program that gives users a point-to-point way to connect and interact with other users. "We want to model the way humans think and feel," Cheng says. "Youíd think the computer was invented 30 or 50 years ago, or whenever, to help you do all the tasks you hate, and then you could go out in the real world and play and have fun. Thatís the best example of how people have adapted the technology. They actually now use it to have fun and communicate with people, almost more than anything else."

Down the hall, research work of a completely different sort is going on. In contrast to Wilsonís office, Bill Dolanís area is stark. No art on the walls, no crazy distractions. Just a computer and Dolan. He applies himself to the study of Natural Language Processing, and helping computers understand the ambiguity of words. He and his team study language and the potential for software to analyze it, understand it and generate it naturally. Itís no easy task.

Thereís a certain irony that language is relatively easy for humans to learn, but extremely hard for a computer to grasp, as its basic language is ones and zeroes. "This whole data-driven translation thing comes out of the Rosetta stone," Dolan says with a chuckle, "where there were different versions of the same Egyptian legal decree. Itís just an updated version of that same technique of trying to identify linear A from linear B."

And none of it is possible without deep, meaningful research, which is the companyís creative juice. "Research here," says Cheng, "is great. What I love is that we have a small group with a mix of skills--researchers, designers, developers, testers and social scientists. Research is free to do what they think is important. We have a lot of motivation to talk to product groups or do things with a lot of impact. For me itís been a great environment. And I think the products reflect that."

THE LOVE OF RESEARCH Microsoft was a natural fit for Andy Wilson. The Philadelphia native was consumed by computers from a young age. In high school, his big interests initially were art and architecture. It was at a summer architecture program that he got his brain around an early computer-aided design (CAD) program that lit him up for the potential of computing. "I was really blown away by that," says the affable redhead. "I ended up getting an Apple II at home, and really getting into all of that stuff."

"That stuff" would be computer science. Wilson attended Cornell University, then graduate school at MIT. He focused on computer vision, using cameras to sense and understand the world. "Until then," he says, "people were using computer vision to find tanks, and satellites and missile silos. Around that time computers were getting faster and cheap enough that you could start thinking that maybe computers could understand the real world in a real time situation."

One of the things that appealed to Wilson about the corporate environment was that he could do significant research and also have an impact on products that might end up in the real world. "Thereís a lot of interesting technology," says Wilson, whoís one of 12 researchers in ASI. "But how do we get there from here? How do we make the transition from desktop-centric computing to the next level, where things are everywhere in your environment? Computing is literally embedded in the walls. You canít possibly sit in front of every computer you have in your house right now. Whatís it going to be like in the future when thereís 10 times as much computation in your house, in your car, in your hotel room?"

As we move to and from these little islands of computing--including popular endpoints like cell phones--there is much "interesting stuff" to be done, says Wilson. How will we interact? Speech is one possibility, but not always the best way to interface. Hence his work on gesture recognition. If you give a system a bit of awareness about whatís going on, which can include the machine simply sensing your presence, and possibly knowing whatever it was you were working on previously, or any of a host of awarenesses, it will be of much greater value.

The creative environment Wilson enjoys at Microsoft affords him the time and resources to chase those problems without the constraints of market viability. Itís a creative place. "Weíve got a lot of great, creative people, from all over the map in terms of background and experience," he says, comparing the environment to grad school. "In many ways, itís better. You have the flexibility to work on multiple projects. Itís easy to sit around a lunch counter and have a great idea and say, ĎLetís do it.í The cool thing is that we have enough people and enough mass that itís a deep bench, people from all parts of computer science, theoretical, crypto, UI [User Interface], to graphics, even a hardware group. Thereís a lot of variety."

Nobody in his group, he says, has to write business plans. He does, however, get requests periodically by the Microsoft products group, which needs his help with a particular application or function, and for which there are specific deadlines. He gave a talk on the Microsoft campus recently, and the next day, his e-mail box was full of new contacts. But for the most part, he is able to work on the speculative edge, and by virtue of that, able to impact company strategies for the future.

"I want to change the way people relate to their machines," Wilson says with a broad smile. "In 20 years, I donít want to be sitting in front of my PC 10 hours a day. How do we make the time we spend with our machines fit in with the way we do things?"

Itís a big question. But Andy Wilson, Lili Cheng, Bill Dolan and the hundreds of other Microsoft researchers are probing all those ones and zeroes for the a nswers. For users like us, what they discover will fundamentally change the way we compute, or perhaps more accurately, change the way computers "think"-- more like us.

Don Campbell is a frequent contributor to World Traveler magazine.
Website design by The Flower Press