TU Darmstadt / ULB / TUprints

ExGenNet: Learning to Generate Robotic Facial Expression Using Facial Expression Recognition

Rawal, Niyati ; Koert, Dorothea ; Turan, Cigdem ; Kersting, Kristian ; Peters, Jan ; Stock-Homburg, Ruth (2022)
ExGenNet: Learning to Generate Robotic Facial Expression Using Facial Expression Recognition.
In: Frontiers in Robotics and AI, 2022, 8
doi: 10.26083/tuprints-00020336
Article, Secondary publication, Publisher's Version

[img] Text
frobt-08-730317.pdf
Copyright Information: CC BY 4.0 International - Creative Commons, Attribution.

Download (2MB)
Item Type: Article
Type of entry: Secondary publication
Title: ExGenNet: Learning to Generate Robotic Facial Expression Using Facial Expression Recognition
Language: English
Date: 13 May 2022
Place of Publication: Darmstadt
Year of primary publication: 2022
Publisher: Frontiers Media S.A.
Journal or Publication Title: Frontiers in Robotics and AI
Volume of the journal: 8
Collation: 11 Seiten
DOI: 10.26083/tuprints-00020336
Corresponding Links:
Origin: Secondary publication DeepGreen
Abstract:

The ability of a robot to generate appropriate facial expressions is a key aspect of perceived sociability in human-robot interaction. Yet many existing approaches rely on the use of a set of fixed, preprogrammed joint configurations for expression generation. Automating this process provides potential advantages to scale better to different robot types and various expressions. To this end, we introduce ExGenNet, a novel deep generative approach for facial expressions on humanoid robots. ExGenNets connect a generator network to reconstruct simplified facial images from robot joint configurations with a classifier network for state-of-the-art facial expression recognition. The robots’ joint configurations are optimized for various expressions by backpropagating the loss between the predicted expression and intended expression through the classification network and the generator network. To improve the transfer between human training images and images of different robots, we propose to use extracted features in the classifier as well as in the generator network. Unlike most studies on facial expression generation, ExGenNets can produce multiple configurations for each facial expression and be transferred between robots. Experimental evaluations on two robots with highly human-like faces, Alfie (Furhat Robot) and the android robot Elenoide, show that ExGenNet can successfully generate sets of joint configurations for predefined facial expressions on both robots. This ability of ExGenNet to generate realistic facial expressions was further validated in a pilot study where the majority of human subjects could accurately recognize most of the generated facial expressions on both the robots.

Uncontrolled Keywords: facial expression generation, humanoid robots, facial expression recognition, neural networks, gradient descent
Status: Publisher's Version
URN: urn:nbn:de:tuda-tuprints-203368
Classification DDC: 000 Generalities, computers, information > 004 Computer science
Divisions: 01 Department of Law and Economics > Betriebswirtschaftliche Fachgebiete > Department of Marketing & Human Resource Management
20 Department of Computer Science > Intelligent Autonomous Systems
20 Department of Computer Science > Artificial Intelligence and Machine Learning
Forschungsfelder > Information and Intelligence > Cognitive Science
Zentrale Einrichtungen > hessian.AI - The Hessian Center for Artificial Intelligence
Date Deposited: 13 May 2022 13:20
Last Modified: 14 Nov 2023 19:04
SWORD Depositor: Deep Green
URI: https://tuprints.ulb.tu-darmstadt.de/id/eprint/20336
PPN: 499683943
Export:
Actions (login required)
View Item View Item