Log In Sign Up

Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulation

by   Stephen James, et al.

Despite the success of reinforcement learning methods, they have yet to have their breakthrough moment when applied to a broad range of robotic manipulation tasks. This is partly due to the fact that reinforcement learning algorithms are notoriously difficult and time consuming to train, which is exacerbated when training from images rather than full-state inputs. As humans perform manipulation tasks, our eyes closely monitor every step of the process with our gaze focusing sequentially on the objects being manipulated. With this in mind, we present our Attention-driven Robotic Manipulation (ARM) algorithm, which is a general manipulation algorithm that can be applied to a range of sparse-rewarded tasks, given only a small number of demonstrations. ARM splits the complex task of manipulation into a 3 stage pipeline: (1) a Q-attention agent extracts interesting pixel locations from RGB and point cloud inputs, (2) a next-best pose agent that accepts crops from the Q-attention agent and outputs poses, and (3) a control agent that takes the goal pose and outputs joint actions. We show that current learning algorithms fail on a range of RLBench tasks, whilst ARM is successful.


page 2

page 4

page 7

page 9


Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisation

Reflecting on the last few years, the biggest breakthroughs in deep rein...

An Open-Source Multi-Goal Reinforcement Learning Environment for Robotic Manipulation with Pybullet

This work re-implements the OpenAI Gym multi-goal robotic manipulation e...

Towards a Sample Efficient Reinforcement Learning Pipeline for Vision Based Robotics

Deep Reinforcement learning holds the guarantee of empowering self-rulin...

An Empowerment-based Solution to Robotic Manipulation Tasks with Sparse Rewards

In order to provide adaptive and user-friendly solutions to robotic mani...

Learning to Centralize Dual-Arm Assembly

Even though industrial manipulators are widely used in modern manufactur...

Deictic Image Maps: An Abstraction For Learning Pose Invariant Manipulation Policies

In applications of deep reinforcement learning to robotics, it is often ...

Active Inference for Robotic Manipulation

Robotic manipulation stands as a largely unsolved problem despite signif...