Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization

05/25/2022
by   Sungryull Sohn, et al.
1

We tackle real-world problems with complex structures beyond the pixel-based game or simulator. We formulate it as a few-shot reinforcement learning problem where a task is characterized by a subtask graph that defines a set of subtasks and their dependencies that are unknown to the agent. Different from the previous meta-rl methods trying to directly infer the unstructured task embedding, our multi-task subtask graph inferencer (MTSGI) first infers the common high-level task structure in terms of the subtask graph from the training tasks, and use it as a prior to improve the task inference in testing. Our experiment results on 2D grid-world and complex web navigation domains show that the proposed method can learn and leverage the common underlying structure of the tasks for faster adaptation to the unseen tasks than various existing algorithms such as meta reinforcement learning, hierarchical reinforcement learning, and other heuristic agents.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/01/2020

Meta Reinforcement Learning with Autonomous Inference of Subtask Dependencies

We propose and address a novel few-shot RL problem, where a task is char...
research
03/28/2022

Learning Parameterized Task Structure for Generalization to Unseen Entities

Real world tasks are hierarchical and compositional. Tasks can be compos...
research
02/06/2023

Robust Subtask Learning for Compositional Generalization

Compositional reinforcement learning is a promising approach for trainin...
research
08/07/2022

A Game-Theoretic Perspective of Generalization in Reinforcement Learning

Generalization in reinforcement learning (RL) is of importance for real ...
research
02/09/2019

Meta-Curvature

We propose to learn curvature information for better generalization and ...
research
09/14/2021

Few-shot Quality-Diversity Optimisation

In the past few years, a considerable amount of research has been dedica...
research
09/14/2017

Autonomous Extracting a Hierarchical Structure of Tasks in Reinforcement Learning and Multi-task Reinforcement Learning

Reinforcement learning (RL), while often powerful, can suffer from slow ...

Please sign up or login with your details

Forgot password? Click here to reset