Causal Transfer for Imitation Learning and Decision Making under Sensor-shift

03/02/2020
by   Jalal Etesami, et al.
0

Learning from demonstrations (LfD) is an efficient paradigm to train AI agents. But major issues arise when there are differences between (a) the demonstrator's own sensory input, (b) our sensors that observe the demonstrator and (c) the sensory input of the agent we train. In this paper, we propose a causal model-based framework for transfer learning under such "sensor-shifts", for two common LfD tasks: (1) inferring the effect of the demonstrator's actions and (2) imitation learning. First we rigorously analyze, on the population-level, to what extent the relevant underlying mechanisms (the action effects and the demonstrator policy) can be identified and transferred from the available observations together with prior knowledge of sensor characteristics. And we device an algorithm to infer these mechanisms. Then we introduce several proxy methods which are easier to calculate, estimate from finite data and interpret than the exact solutions, alongside theoretical bounds on their closeness to the exact ones. We validate our two main methods on simulated and semi-real world data.

READ FULL TEXT

page 1

page 2

page 3

page 4

06/22/2022

Imitation Learning for Generalizable Self-driving Policy with Sim-to-real Transfer

Imitation Learning uses the demonstrations of an expert to uncover the o...
10/02/2019

Task-Relevant Adversarial Imitation Learning

We show that a critical problem in adversarial imitation from high-dimen...
08/12/2022

Sequential Causal Imitation Learning with Unobserved Confounders

"Monkey see monkey do" is an age-old adage, referring to naïve imitation...
05/24/2018

Inverse POMDP: Inferring What You Think from What You Do

Complex behaviors are often driven by an internal model, which integrate...
05/22/2018

Maximum Causal Tsallis Entropy Imitation Learning

In this paper, we propose a novel maximum causal Tsallis entropy (MCTE) ...
10/05/2019

Making sense of sensory input

This paper attempts to answer a central question in unsupervised learnin...
12/07/2021

Causal Imitative Model for Autonomous Driving

Imitation learning is a powerful approach for learning autonomous drivin...