Deep Intrinsically Motivated Continuous Actor-Critic for Efficient Robotic Visuomotor Skill Learning

10/26/2018
by   Muhammad Burhan Hafez, et al.
6

In this paper, we present a new intrinsically motivated actor-critic algorithm for learning continuous motor skills directly from raw visual input. Our neural architecture is composed of a critic and an actor network. Both networks receive the hidden representation of a deep convolutional autoencoder which is trained to reconstruct the visual input, while the centre-most hidden representation is also optimized to estimate the state value. Separately, an ensemble of predictive world models generates, based on its learning progress, an intrinsic reward signal which is combined with the extrinsic reward to guide the exploration of the actor-critic learner. Our approach is more data- efficient and inherently more stable than the existing actor-critic methods for continuous control from pixel data. We evaluate our algorithm for the task of learning robotic reaching and grasping skills on a realistic physics simulator and on a humanoid robot. The results show that the control policies learned with our approach can achieve better performance than the compared state-of-the-art and baseline algorithms in both dense-reward and challenging sparse-reward settings.

READ FULL TEXT

page 8

page 10

page 13

page 15

page 16

page 18

page 19

research
07/18/2016

A Batch, Off-Policy, Actor-Critic Algorithm for Optimizing the Average Reward

We develop an off-policy actor-critic algorithm for learning an optimal ...
research
05/20/2023

Off-Policy Average Reward Actor-Critic with Deterministic Policy Search

The average reward criterion is relatively less studied as most existing...
research
11/03/2020

Intrinsic Robotic Introspection: Learning Internal States From Neuron Activations

We present an introspective framework inspired by the process of how hum...
research
04/10/2019

Actor-Critic Instance Segmentation

Most approaches to visual scene analysis have emphasised parallel proces...
research
12/17/2021

Symmetry-aware Neural Architecture for Embodied Visual Navigation

Visual exploration is a task that seeks to visit all the navigable areas...
research
12/20/2022

Hybrid Rule-Neural Coreference Resolution System based on Actor-Critic Learning

A coreference resolution system is to cluster all mentions that refer to...

Please sign up or login with your details

Forgot password? Click here to reset