Donate to Science & Enterprise

S&E on Mastodon

S&E on LinkedIn

S&E on Flipboard

Please share Science & Enterprise

Researchers Give Robotics a Human Face

Mask-bot image (Uli Benz,TU Muenchen)

Mask-bot image (Uli Benz,TU Muenchen)

Research engineers at Technical University of Munich (Technische Universitaet Muenchen, TUM) in Germany and the National Institute of Advanced Industrial Science and Technology in Japan have developed a way to give robots a human-like plastic head. Called Mask-bot, the technology may have a more immediate application as a tool to create avatars for participants in live videoconferences.

The solution uses a projector to beam the 3D image of a face onto the back of a plastic mask, and a computer to control voice and facial expressions. The projector positioned behind the mask beams a human face — that can be changed on demand — onto the back of the mask, creating realistic features that can be seen from various angles, including the side.

Mask-bot functions without requiring a video image of the person speaking, which enables the use of the image as the face of a robot. A separate program enables the system to convert a normal two-dimensional photograph into a correctly proportioned projection for a three-dimensional mask.

Further algorithms provide the facial expressions and voice. To replicate facial expressions, TUM’s Takaaki Kuratate developed a talking head animation engine, where a computer filters a series of face motion data from people collected by a motion capture system. The system then selects the facial expressions that best match a specific sound or phoneme when it is being spoken.

The animation software extracts a set of facial coordinates from each of these expressions, which it can then assign to any new face. Emotion synthesis software delivers the visible emotional nuances that indicate, for example, when someone is happy, sad or angry.

Mask-bot can already reproduce simple dialog. When Kuratate says “rainbow”, for example, Mask-bot responds with an elaborate sentence on the subject: “When the sunlight strikes raindrops in the air, they act like a prism and form a rainbow.” And when it talks, Mask-bot also moves its head a little and raises its eyebrows to create a knowledgeable impression.

The researchers say Mask-bot can reproduce content typed via a keyboard, now in English and Japanese, and soon German. A text-to-speech system converts text to audio signals, producing a female or male voice, which can then set to quiet or loud, happy or sad. But Mask-bot is not yet able to understand much of the spoken word, only listen and make appropriate responses as part of a fixed programming sequence.

As a robotics module, Mask-bot is still a prototype, but it could see more immediate commercial use in video conferencing. With Mask-bot, says Kuratate, “you can create a realistic replica of a person that actually sits and speaks with you at the conference table. You can use a generic mask for male and female, or you can provide a custom-made mask for each person.”

The following short video gives a live demonstration of Mask-bot.

 

Read more: Robot Being Developed to Aid in Care of Elderly

*     *     *

2 comments to Researchers Give Robotics a Human Face