Wearable Personal Instruction System Being Developed

Grad student Zhou Chen uses Gabriel and Google Glass with freehand sketching (Carnegie Mellon University)

2 December 2015. A device that provides personal step-by-step instruction to individuals as they undertake various tasks is being developed by computer scientists at Carnegie Mellon University. The system, designed in the lab of professor Mahadev Satyanarayanan, is funded by a four-year $2.8 million grant from National Science Foundation.

Satyanarayanan and colleagues are seeking a way to provide problem-solving advice as individuals encounter day-to-day tasks at work or home. Their system, called Gabriel after the angel who appears in the Old and New Testament delivering messages at the behest of God, aims to deliver more routine messages to users from scaled-down data collections called cloudlets, rather than a supreme being.

Gabriel will combine wearable computing technology, such as Google Glass, with cloudlets, and take advantage of advances in computer vision, robotics, and cognitive algorithms, like those employed by IBM’s Watson supercomputer to recognize objects and understand their context. From robotics, Gabriel’s developers are including sensing and task planning functions, but unlike robots, will be activated by people on demand rather than autonomously.

“The experience is much like a driver using a GPS navigation system,” says Satyanarayanan in a university statement. “It gives you instructions when you need them, corrects you when you make a mistake and, most of the time, shuts up so it doesn’t bug you.”

Gabriel’s developers say cloudlets make possible the system’s speed and agility. Cloudlets will store data needed to support Gabriel with the computational power of the cloud, but situated closer to the group of mobile users, such as at a nearby cell tower, to minimize the connections needed to send and receive data and interact with wearable devices. Reducing data transfer times to a few milliseconds will allow for almost instantaneous real-time advice or instructions for even activities as fast-paced as teaching ping-pong.

Satyanarayanan’s lab developed proof-of-concept versions of Gabriel to coach ping-pong, but also to build lego structures, and teach freehand sketching. These versions were built with Open Stack, open-source software for creating cloud platforms, and available from Satyanarayanan’s lab.

The NSF grant supports work in the underlying technologies of Gabriel, including computer vision and location sensing, with initial applications aimed at tasks requiring specialized knowledge and skills.

Read more:

• Crop Disease Images Collected, Available Via Smartphones
• Sanofi, Google to Partner on Diabetes Care, Devices
• NSF Funds Math Tutoring Software Commercialization
• University Campus Serving as Internet-of-Things Lab
• Internet-of-Things Security Architecture Designed

*     *     *