Chalvatzaki, Georgia ; Younes, Ali ; Nandha, Daljeet ; Le, An Thai ; Ribeiro, Leonardo F. R. ; Gurevych, Iryna (2023)
Learning to reason over scene graphs: a case study of finetuning GPT-2 into a robot language model for grounded task planning.
In: Frontiers in Robotics and AI, 2023, 10
doi: 10.26083/tuprints-00024479
Article, Secondary publication, Publisher's Version
Text
frobt-10-1221739.pdf Copyright Information: CC BY 4.0 International - Creative Commons, Attribution. Download (35MB) |
Item Type: | Article |
---|---|
Type of entry: | Secondary publication |
Title: | Learning to reason over scene graphs: a case study of finetuning GPT-2 into a robot language model for grounded task planning |
Language: | English |
Date: | 11 September 2023 |
Place of Publication: | Darmstadt |
Year of primary publication: | 2023 |
Publisher: | Frontiers Media S.A. |
Journal or Publication Title: | Frontiers in Robotics and AI |
Volume of the journal: | 10 |
Collation: | 15 Seiten |
DOI: | 10.26083/tuprints-00024479 |
Corresponding Links: | |
Origin: | Secondary publication DeepGreen |
Abstract: | Long-horizon task planning is essential for the development of intelligent assistive and service robots. In this work, we investigate the applicability of a smaller class of large language models (LLMs), specifically GPT-2, in robotic task planning by learning to decompose tasks into subgoal specifications for a planner to execute sequentially. Our method grounds the input of the LLM on the domain that is represented as a scene graph, enabling it to translate human requests into executable robot plans, thereby learning to reason over long-horizon tasks, as encountered in the ALFRED benchmark. We compare our approach with classical planning and baseline methods to examine the applicability and generalizability of LLM-based planners. Our findings suggest that the knowledge stored in an LLM can be effectively grounded to perform long-horizon task planning, demonstrating the promising potential for the future application of neuro-symbolic planning methods in robotics. |
Uncontrolled Keywords: | robot learning, task planning, grounding, language models (LMs), pretrained models, scene graphs |
Status: | Publisher's Version |
URN: | urn:nbn:de:tuda-tuprints-244795 |
Classification DDC: | 000 Generalities, computers, information > 004 Computer science |
Divisions: | 20 Department of Computer Science > Ubiquitous Knowledge Processing Zentrale Einrichtungen > hessian.AI - The Hessian Center for Artificial Intelligence |
Date Deposited: | 11 Sep 2023 12:35 |
Last Modified: | 30 Oct 2023 07:16 |
SWORD Depositor: | Deep Green |
URI: | https://tuprints.ulb.tu-darmstadt.de/id/eprint/24479 |
PPN: | 512750394 |
Export: |
View Item |