Donate to Science & Enterprise

S&E on Mastodon

S&E on LinkedIn

S&E on Flipboard

Please share Science & Enterprise

Carnegie Mellon, Microsoft Develop Touch-Screen Projection

OmniTouch shoulder assembly (Chris Harrison, Carnegie-Mellon University)

OmniTouch shoulder assembly: camera and projector (Chris Harrison, Carnegie-Mellon University)

Researchers at Carnegie Mellon University in Pittsburgh and Microsoft’s research labs have developed OmniTouch, a device that projects touch screen capability on any flat surface. Chris Harrison, a computer science Ph.D. student and a developer of OmniTouch, will discuss the system on Wednesday at the Association for Computing Machinery’s Symposium on User Interface Software and Technology in Santa Barbara, California.

Harrison, who conducts his research in CMU’s Human-Computer Interaction Institute, began working on OmniTouch while an intern at Microsoft Research. He will present his findings at the symposium with co-developers Hrvoje Benko and Andrew Wilson of Microsoft.

OmniTouch can turn ordinary flat surfaces such as legal pads, walls, or even people’s own hands into interactive surfaces like computer touch screens. The system uses a depth-sensing camera, like the one found on the Microsoft Xbox Kinect, to track the user’s fingers on remote flat surfaces. This capability to project an interactive touch screen allows users to control interactive applications by tapping or dragging their fingers on those projected surfaces.

The user can then interact with the projected touch screen as if it were on a smartphone or tablet computer, superimposing keyboards, keypads and other controls onto the target surface. OmniTouch automatically adjusts for the surface’s shape and orientation to minimize distortion of the projected images.

With OmniTouch, the palm of the hand can be used as a phone keypad, or as a tablet for jotting down brief notes. Maps projected onto a wall can be panned and zoomed with the same finger motions that work with a conventional touch screen. Moreover, it can track three-dimensional motion and sense when fingers “click” on or hover over a projected object.

Benko, a researcher in Microsoft Research’s Adaptive Systems and Interaction group, says “We see this work as an evolutionary step in a larger effort at Microsoft Research to investigate the unconventional use of touch and gesture in devices to extend our vision of ubiquitous computing even further.”

The system now requires the depth-sensing camera and laser pico-projector to be mounted on a user’s shoulder (pictured at top). Harrison anticipates miniaturizing these devices to the size of a deck of cards or even a matchbox, to make them easier to wear.

This video shows more capabilities of OmniTouch and gives results of user evaluation tests.

 

Read more: Radar in Shoes Can Step In When GPS Fails

*     *     *

Comments are closed.