On the Guaranteed Almost Equivalence between Imitation Learning from Observation and Demonstration

10/16/2020
by   Zhihao Cheng, et al.
0

Imitation learning from observation (LfO) is more preferable than imitation learning from demonstration (LfD) due to the nonnecessity of expert actions when reconstructing the expert policy from the expert data. However, previous studies imply that the performance of LfO is inferior to LfD by a tremendous gap, which makes it challenging to employ LfO in practice. By contrast, this paper proves that LfO is almost equivalent to LfD in the deterministic robot environment, and more generally even in the robot environment with bounded randomness. In the deterministic robot environment, from the perspective of the control theory, we show that the inverse dynamics disagreement between LfO and LfD approaches zero, meaning that LfO is almost equivalent to LfD. To further relax the deterministic constraint and better adapt to the practical environment, we consider bounded randomness in the robot environment and prove that the optimizing targets for both LfD and LfO remain almost same in the more generalized setting. Extensive experiments for multiple robot tasks are conducted to empirically demonstrate that LfO achieves comparable performance to LfD. In fact, most common robot systems in reality are the robot environment with bounded randomness (i.e., the environment this paper considered). Hence, our findings greatly extend the potential of LfO and suggest that we can safely apply LfO without sacrificing the performance compared to LfD in practice.

READ FULL TEXT

page 8

page 18

research
06/18/2019

RIDM: Reinforced Inverse Dynamics Modeling for Learning from a Single Observed Demonstration

Imitation learning has long been an approach to alleviate the tractabili...
research
05/23/2023

Learning from demonstrations: An intuitive VR environment for imitation learning of construction robots

Construction robots are challenging the traditional paradigm of labor in...
research
09/27/2019

Zero-shot Imitation Learning from Demonstrations for Legged Robot Visual Navigation

Imitation learning is a popular approach for training effective visual n...
research
05/22/2019

Imitation Learning from Video by Leveraging Proprioception

Classically, imitation learning algorithms have been developed for ideal...
research
10/10/2019

Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement

This paper studies Learning from Observations (LfO) for imitation learni...
research
08/02/2021

Self-Supervised Disentangled Representation Learning for Third-Person Imitation Learning

Humans learn to imitate by observing others. However, robot imitation le...
research
07/11/2019

Graph-Structured Visual Imitation

We cast visual imitation as a visual correspondence problem. Our robotic...

Please sign up or login with your details

Forgot password? Click here to reset