Inverse Reinforcement Learning via Nonparametric Spatio-Temporal Subgoal Modeling

by   Adrian Šošić, et al.

Recent advances in the field of inverse reinforcement learning (IRL) have yielded sophisticated frameworks which relax the original modeling assumption that the behavior of an observed agent reflects only a single intention. Instead, the demonstration data is typically divided into parts, to account for the fact that different trajectories may correspond to different intentions, e.g., because they were generated by different domain experts. In this work, we go one step further: using the intuitive concept of subgoals, we build upon the premise that even a single trajectory can be explained more efficiently locally within a certain context than globally, enabling a more compact representation of the observed behavior. Based on this assumption, we build an implicit intentional model of the agent's goals to forecast its behavior in unobserved situations. The result is an integrated Bayesian prediction framework which provides smooth policy estimates that are consistent with the expert's plan and significantly outperform existing IRL solutions. Most notably, our framework naturally handles situations where the intentions of the agent change with time and classical IRL algorithms fail. In addition, due to its probabilistic nature, the model can be straightforwardly applied in an active learning setting to guide the demonstration process of the expert.



page 14

page 25

page 28

page 30

page 34

page 36


Active Learning for Risk-Sensitive Inverse Reinforcement Learning

One typical assumption in inverse reinforcement learning (IRL) is that h...

RIDM: Reinforced Inverse Dynamics Modeling for Learning from a Single Observed Demonstration

Imitation learning has long been an approach to alleviate the tractabili...

Active Deep Q-learning with Demonstration

Recent research has shown that although Reinforcement Learning (RL) can ...

Neuroevolution-Based Inverse Reinforcement Learning

The problem of Learning from Demonstration is targeted at learning to pe...

Risk-Aware Active Inverse Reinforcement Learning

Active learning from demonstration allows a robot to query a human for s...

Multi-class Generalized Binary Search for Active Inverse Reinforcement Learning

This paper addresses the problem of learning a task from demonstration. ...

Inverse Reinforcement Learning from Incomplete Observation Data

Inverse reinforcement learning (IRL) aims to explain observed strategic ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.