TU Darmstadt / ULB / TUprints

Multi-Channel Interactive Reinforcement Learning for Sequential Tasks

Koert, Dorothea ; Kircher, Maximilian ; Salikutluk, Vildan ; D'Eramo, Carlo ; Peters, Jan (2021):
Multi-Channel Interactive Reinforcement Learning for Sequential Tasks. (Publisher's Version)
In: Frontiers in Robotics and AI, 7, Frontiers, e-ISSN 2296-9144,
DOI: 10.26083/tuprints-00019239,

Copyright Information: CC BY 4.0 International - Creative Commons, Attribution.

Download (3MB) | Preview
Item Type: Article
Origin: Secondary publication via sponsored Golden Open Access
Status: Publisher's Version
Title: Multi-Channel Interactive Reinforcement Learning for Sequential Tasks
Language: English

The ability to learn new tasks by sequencing already known skills is an important requirement for future robots. Reinforcement learning is a powerful tool for this as it allows for a robot to learn and improve on how to combine skills for sequential tasks. However, in real robotic applications, the cost of sample collection and exploration prevent the application of reinforcement learning for a variety of tasks. To overcome these limitations, human input during reinforcement can be beneficial to speed up learning, guide the exploration and prevent the choice of disastrous actions. Nevertheless, there is a lack of experimental evaluations of multi-channel interactive reinforcement learning systems solving robotic tasks with input from inexperienced human users, in particular for cases where human input might be partially wrong. Therefore, in this paper, we present an approach that incorporates multiple human input channels for interactive reinforcement learning in a unified framework and evaluate it on two robotic tasks with 20 inexperienced human subjects. To enable the robot to also handle potentially incorrect human input we incorporate a novel concept for self-confidence, which allows the robot to question human input after an initial learning phase. The second robotic task is specifically designed to investigate if this self-confidence can enable the robot to achieve learning progress even if the human input is partially incorrect. Further, we evaluate how humans react to suggestions of the robot, once the robot notices human input might be wrong. Our experimental evaluations show that our approach can successfully incorporate human input to accelerate the learning process in both robotic tasks even if it is partially wrong. However, not all humans were willing to accept the robot's suggestions or its questioning of their input, particularly if they do not understand the learning process and the reasons behind the robot's suggestions. We believe that the findings from this experimental evaluation can be beneficial for the future design of algorithms and interfaces of interactive reinforcement learning systems used by inexperienced users.

Journal or Publication Title: Frontiers in Robotics and AI
Volume of the journal: 7
Place of Publication: Darmstadt
Publisher: Frontiers
Collation: 19 Seiten
Classification DDC: 000 Allgemeines, Informatik, Informationswissenschaft > 004 Informatik
Divisions: 20 Department of Computer Science > Intelligent Autonomous Systems
Date Deposited: 04 Aug 2021 08:03
Last Modified: 24 Jun 2022 18:02
DOI: 10.26083/tuprints-00019239
Corresponding Links:
URN: urn:nbn:de:tuda-tuprints-192395
URI: https://tuprints.ulb.tu-darmstadt.de/id/eprint/19239
Actions (login required)
View Item View Item