TU Darmstadt / ULB / TUprints

Robust Reinforcement Learning: A Review of Foundations and Recent Advances

Moos, Janosch ; Hansel, Kay ; Abdulsamad, Hany ; Stark, Svenja ; Clever, Debora ; Peters, Jan (2022):
Robust Reinforcement Learning: A Review of Foundations and Recent Advances. (Publisher's Version)
In: Machine Learning and Knowledge Extraction, 4 (1), pp. 276-315. MDPI, e-ISSN 2504-4990,
DOI: 10.26083/tuprints-00021118,
[Article]

[img] Text
make-04-00013-v2.pdf
Available under: CC BY 4.0 International - Creative Commons, Attribution.

Download (1MB)
Item Type: Article
Origin: Secondary publication DeepGreen
Status: Publisher's Version
Title: Robust Reinforcement Learning: A Review of Foundations and Recent Advances
Language: English
Abstract:

Reinforcement learning (RL) has become a highly successful framework for learning in Markov decision processes (MDP). Due to the adoption of RL in realistic and complex environments, solution robustness becomes an increasingly important aspect of RL deployment. Nevertheless, current RL algorithms struggle with robustness to uncertainty, disturbances, or structural changes in the environment. We survey the literature on robust approaches to reinforcement learning and categorize these methods in four different ways: (i) Transition robust designs account for uncertainties in the system dynamics by manipulating the transition probabilities between states; (ii) Disturbance robust designs leverage external forces to model uncertainty in the system behavior; (iii) Action robust designs redirect transitions of the system by corrupting an agent’s output; (iv) Observation robust designs exploit or distort the perceived system state of the policy. Each of these robust designs alters a different aspect of the MDP. Additionally, we address the connection of robustness to the risk-based and entropy-regularized RL formulations. The resulting survey covers all fundamental concepts underlying the approaches to robust reinforcement learning and their recent advances.

Journal or Publication Title: Machine Learning and Knowledge Extraction
Volume of the journal: 4
Issue Number: 1
Publisher: MDPI
Uncontrolled Keywords: reinforcement learning, robustness, min-max optimization
Classification DDC: 000 Allgemeines, Informatik, Informationswissenschaft > 004 Informatik
600 Technik, Medizin, angewandte Wissenschaften > 620 Ingenieurwissenschaften
Divisions: 16 Department of Mechanical Engineering > Institute for Mechatronic Systems in Mechanical Engineering (IMS)
20 Department of Computer Science > Intelligent Autonomous Systems
Date Deposited: 11 Apr 2022 11:34
Last Modified: 11 Apr 2022 11:34
DOI: 10.26083/tuprints-00021118
Corresponding Links:
URN: urn:nbn:de:tuda-tuprints-211188
SWORD Depositor: Deep Green
URI: https://tuprints.ulb.tu-darmstadt.de/id/eprint/21118
PPN:
Export:
Actions (login required)
View Item View Item