Adversarial Skill Networks: Unsupervised Robot Skill Learning from Video

10/21/2019
by   Oier Mees, et al.
23

Key challenges for the deployment of reinforcement learning (RL) agents in the real world are the discovery, representation and reuse of skills in the absence of a reward function. To this end, we propose a novel approach to learn a task-agnostic skill embedding space from unlabeled multi-view videos. Our method learns a general skill embedding independently from the task context by using an adversarial loss. We combine a metric learning loss, which utilizes temporal video coherence to learn a state representation, with an entropy regularized adversarial skill-transfer loss. The metric learning loss learns a disentangled representation by attracting simultaneous viewpoints of the same observations and repelling visually similar frames from temporal neighbors. The adversarial skill-transfer loss enhances re-usability of learned skill embeddings over multiple task domains. We show that the learned embedding enables training of continuous control policies to solve novel tasks that require the interpolation of previously seen skills. Our extensive evaluation with both simulation and real world data demonstrates the effectiveness of our method in learning transferable skills from unlabeled interaction videos and composing them for new tasks.

READ FULL TEXT

page 1

page 3

page 4

page 6

research
10/22/2020

Accelerating Reinforcement Learning with Learned Skill Priors

Intelligent agents rely heavily on prior experience when learning a new ...
research
12/20/2017

Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning

Learning policies for complex tasks that require multiple different skil...
research
01/27/2022

Rethinking Learning Dynamics in RL using Adversarial Networks

We present a learning mechanism for reinforcement learning of closely re...
research
06/21/2019

Disentangled Skill Embeddings for Reinforcement Learning

We propose a novel framework for multi-task reinforcement learning (MTRL...
research
01/09/2019

Manipulation-skill Assessment from Videos with Spatial Attention Network

Recent advances in computer vision have made it possible to automaticall...
research
12/13/2018

The Pros and Cons: Rank-aware Temporal Attention for Skill Determination in Long Videos

We present a new model to determine relative skill from long videos, thr...
research
11/25/2021

Robot Skill Adaptation via Soft Actor-Critic Gaussian Mixture Models

A core challenge for an autonomous agent acting in the real world is to ...

Please sign up or login with your details

Forgot password? Click here to reset