TU Darmstadt / ULB / TUprints

Interactively learning behavior trees from imperfect human demonstrations

Scherf, Lisa ; Schmidt, Aljoscha ; Pal, Suman ; Koert, Dorothea (2023)
Interactively learning behavior trees from imperfect human demonstrations.
In: Frontiers in Robotics and AI, 2023, 10
doi: 10.26083/tuprints-00024370
Article, Secondary publication, Publisher's Version

[img] Text
Copyright Information: CC BY 4.0 International - Creative Commons, Attribution.

Download (41MB)
Item Type: Article
Type of entry: Secondary publication
Title: Interactively learning behavior trees from imperfect human demonstrations
Language: English
Date: 4 August 2023
Place of Publication: Darmstadt
Year of primary publication: 2023
Publisher: Frontiers Media S.A.
Journal or Publication Title: Frontiers in Robotics and AI
Volume of the journal: 10
Collation: 19 Seiten
DOI: 10.26083/tuprints-00024370
Corresponding Links:
Origin: Secondary publication DeepGreen

Introduction: In Interactive Task Learning (ITL), an agent learns a new task through natural interaction with a human instructor. Behavior Trees (BTs) offer a reactive, modular, and interpretable way of encoding task descriptions but have not yet been applied a lot in robotic ITL settings. Most existing approaches that learn a BT from human demonstrations require the user to specify each action step-by-step or do not allow for adapting a learned BT without the need to repeat the entire teaching process from scratch.

Method: We propose a new framework to directly learn a BT from only a few human task demonstrations recorded as RGB-D video streams. We automatically extract continuous pre- and post-conditions for BT action nodes from visual features and use a Backchaining approach to build a reactive BT. In a user study on how non-experts provide and vary demonstrations, we identify three common failure cases of an BT learned from potentially imperfect initial human demonstrations. We offer a way to interactively resolve these failure cases by refining the existing BT through interaction with a user over a web-interface. Specifically, failure cases or unknown states are detected automatically during the execution of a learned BT and the initial BT is adjusted or extended according to the provided user input.

Evaluation and results: We evaluate our approach on a robotic trash disposal task with 20 human participants and demonstrate that our method is capable of learning reactive BTs from only a few human demonstrations and interactively resolving possible failure cases at runtime.

Uncontrolled Keywords: human-robot interaction, interactive task learning, behavior trees, learning from demonstration, robotic tasks, user studies, failure detection, failure recovery
Status: Publisher's Version
URN: urn:nbn:de:tuda-tuprints-243701
Classification DDC: 000 Generalities, computers, information > 004 Computer science
Divisions: 20 Department of Computer Science > Intelligent Autonomous Systems
Zentrale Einrichtungen > Centre for Cognitive Science (CCS)
Date Deposited: 04 Aug 2023 12:04
Last Modified: 19 Oct 2023 13:33
SWORD Depositor: Deep Green
URI: https://tuprints.ulb.tu-darmstadt.de/id/eprint/24370
PPN: 512430977
Actions (login required)
View Item View Item