DeepAI AI Chat
Log In Sign Up

Towards Disturbance-Free Visual Mobile Manipulation

by   Tianwei Ni, et al.

Embodied AI has shown promising results on an abundance of robotic tasks in simulation, including visual navigation and manipulation. The prior work generally pursues high success rates with shortest paths while largely ignoring the problems caused by collision during interaction. This lack of prioritization is understandable: in simulated environments there is no inherent cost to breaking virtual objects. As a result, well-trained agents frequently have catastrophic collision with objects despite final success. In the robotics community, where the cost of collision is large, collision avoidance is a long-standing and crucial topic to ensure that robots can be safely deployed in the real world. In this work, we take the first step towards collision/disturbance-free embodied AI agents for visual mobile manipulation, facilitating safe deployment in real robots. We develop a new disturbance-avoidance methodology at the heart of which is the auxiliary task of disturbance prediction. When combined with a disturbance penalty, our auxiliary task greatly enhances sample efficiency and final performance by knowledge distillation of disturbance into the agent. Our experiments on ManipulaTHOR show that, on testing scenes with novel objects, our method improves the success rate from 61.7 disturbance from 29.8 studies show the value of our pipelined approach. Project site is at


page 1

page 4

page 7

page 17


Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning

Developing a safe and efficient collision avoidance policy for multiple ...

DIViS: Domain Invariant Visual Servoing for Collision-Free Goal Reaching

Robots should understand both semantics and physics to be functional in ...

Flies as Ship Captains? Digital Evolution Unravels Selective Pressures to Avoid Collision in Drosophila

Flies that walk in a covered planar arena on straight paths avoid collid...

ManipulaTHOR: A Framework for Visual Object Manipulation

The domain of Embodied AI has recently witnessed substantial progress, p...

Envelopes and Waves: Safe Multivehicle Collision Avoidance for Horizontal Non-deterministic Turns

We present an approach to analyzing the safety of asynchronous, independ...