Donate to Science & Enterprise

S&E on Mastodon

S&E on LinkedIn

S&E on Flipboard

Please share Science & Enterprise

Google Glass Captioning Developed for Hearing Impaired

James Foley wearing Google Glass

James Foley wearing Google Glass (Georgia Institute of Technology)

3 October 2014. Computer scientists at Georgia Institute of Technology in Atlanta designed a system that converts speech from conversation partners to text, displayed on Google Glass systems worn by people with hearing difficulties. Google Glass is a wearable miniature computer that displays data on eyeglasses worn by the user.

The software is a creation of Georgia Tech’s Contextual Computing Group led by computing professor Thad Starner. The Captioning on Glass system records speech vocalized by conversation partners in a smartphone near the Google Glass wearer. The software then converts the speech to text, transmits the text via Bluetooth, and displays the text on Google Glass.

Google Glass has its own built-in microphone, but that microphone is designed to capture spoken words from the wearer, not by other people in conversation. The separate smartphone, designed to capture speech in telephone calls and spoken commands, records the spoken words of the conversation partner, while minimizing ambient noise that might distort the conversation.

The software, written by graduate student Jay Zuerndorfer, adapts an Android application program interface — a standard set of instructions — for transcription to convert speech of the conversation partner to text. The speech-to-text transcription happens almost instantly, and can be edited by the conversation partner before sending to the Google Glass wearer.

James Foley, a fellow computer science faculty member at Georgia Tech who began losing his hearing, is an early adopter of the system. “This system allows wearers like me to focus on the speaker’s lips and facial gestures,” says Foley in a university statement. “If hard-of-hearing people understand the speech, the conversation can continue immediately without waiting for the caption. However, if I miss a word,” adds Foley, “I can glance at the transcription, get the word or two I need and get back into the conversation.”

Another project from Starner’s lab uses the same basic technology for real-time conversational foreign language translations. Translation on Glass, as the system is called, captures the spoken words of the conversation partner, then translates the words before transmission to Google Glass. The system, still in development, allows for a Google Glass wearer to respond, where the software translates the wearer’s words into the language of the conversation partner. Languages supported so far include English, Spanish, French, Russian, Korean, and Japanese.

Captioning on Glass is not recommended for all conversations, only those with known associates who can work with the smartphone. Google Glass wearers also face occasional resistance from people who find the presence of the devices threatening or intrusive. Starner told the New York Times in May 2013, much of that concern is overblown. “Asocial people will be able to find a way to do asocial things with this technology,” said Starner, “but on average people like to maintain the social contract.”

Foley and Zuerndorfer demonstrate Captioning on Glass in the following video.

Read more:

*     *     *

Comments are closed.