Memory-based gaze prediction in deep imitation learning for robot manipulation

02/10/2022
by   Heecheol Kim, et al.
0

Deep imitation learning is a promising approach that does not require hard-coded control rules in autonomous robot manipulation. The current applications of deep imitation learning to robot manipulation have been limited to reactive control based on the states at the current time step. However, future robots will also be required to solve tasks utilizing their memory obtained by experience in complicated environments (e.g., when the robot is asked to find a previously used object on a shelf). In such a situation, simple deep imitation learning may fail because of distractions caused by complicated environments. We propose that gaze prediction from sequential visual input enables the robot to perform a manipulation task that requires memory. The proposed algorithm uses a Transformer-based self-attention architecture for the gaze estimation based on sequential data to implement memory. The proposed method was evaluated with a real robot multi-object manipulation task that requires memory of the previous states.

READ FULL TEXT

page 2

page 5

page 6

research
08/01/2021

Transformer-based deep imitation learning for dual-arm robot manipulation

Deep imitation learning is promising for solving dexterous manipulation ...
research
02/19/2022

Training Robots without Robots: Deep Imitation Learning for Master-to-Robot Policy Transfer

Deep imitation learning is a promising method for dexterous robot manipu...
research
02/02/2021

Gaze-based dual resolution deep imitation learning for high-precision dexterous robot manipulation

A high-precision manipulation task, such as needle threading, is challen...
research
03/18/2022

Robot peels banana with goal-conditioned dual-action deep imitation learning

A long-horizon dexterous robot manipulation task of deformable objects, ...
research
10/20/2022

VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors

We introduce VIOLA, an object-centric imitation learning approach to lea...
research
02/27/2018

Cloth Manipulation Using Random Forest-Based Controller Parametrization

We present a novel approach for robust manipulation of high-DOF deformab...
research
02/25/2021

Gaze-Informed Multi-Objective Imitation Learning from Human Demonstrations

In the field of human-robot interaction, teaching learning agents from h...

Please sign up or login with your details

Forgot password? Click here to reset