Google Glass can now display captions for hard-of-hearing users

October 3, 2014

Captioning on Glass display captions for the hard-of-hearing (credit: Georgia Tech)

Georgia Institute of Technology researchers have created a speech-to-text Android app for Google Glass that displays captions for hard-of-hearing persons when someone is talking to them in person.

“This system allows wearers like me to focus on the speaker’s lips and facial gestures, “said School of Interactive Computing Professor Jim Foley.

“If hard-of-hearing people understand the speech, the conversation can continue immediately without waiting for the caption. However, if I miss a word, I can glance at the transcription, get the word or two I need and get back into the conversation.”


Georgia Institute of Technology | Captioning on Glass Demo

Foley colleague Professor Thad Starner, who leads the Contextual Computing Group, which is working on the project, says the app has several benefits:

  • Using a smartphone (instead of relying on the Google Glass microphone) reduces background noise and helps eliminate errors.
  • Speakers are more likely to construct their sentences more clearly, avoiding “uhs” and “ums.”
  • A speaker can edit mistakes, sending the changes to the person wearing the device.

“The smartphone uses the Android transcription API to convert the audio to text,” said Jay Zuerndorfer, the Georgia Tech Computer Science graduate student who developed the app. “The text is then streamed to Glass in real time.”

The “Captioning on Glass” app is now available to install from MyGlass. More information here.

Foley and the students are working with the Association of Late Deafened Adults in Atlanta to improve the program. An iPhone app is planned.