DeepAI
Log In Sign Up

Analysing Deep Reinforcement Learning Agents Trained with Domain Randomisation

12/18/2019
by   Tianhong Dai, et al.
44

Deep reinforcement learning has the potential to train robots to perform complex tasks in the real world without requiring accurate models of the robot or its environment. A practical approach is to train agents in simulation, and then transfer them to the real world. One of the most popular methods for achieving this is to use domain randomisation, which involves randomly perturbing various aspects of a simulated environment in order to make trained agents robust to the reality gap between the simulator and the real world. However, less work has gone into understanding such agents-which are deployed in the real world-beyond task performance. In this work we examine such agents, through qualitative and quantitative comparisons between agents trained with and without visual domain randomisation, in order to provide a better understanding of how they function. In this work, we train agents for Fetch and Jaco robots on a visuomotor control task, and evaluate how well they generalise using different unit tests. We tie this with interpretability techniques, providing both quantitative and qualitative data. Finally, we investigate the internals of the trained agents by examining their weights and activations. Our results show that the primary outcome of domain randomisation is more redundant, entangled representations, accompanied with significant statistical/structural changes in the weights; moreover, the types of changes are heavily influenced by the task setup and presence of additional proprioceptive inputs. Furthermore, even with an improved saliency method introduced in this work, we show that qualitative studies may not always correspond with quantitative measures, necessitating the use of a wide suite of inspection tools in order to provide sufficient insights into the behaviour of trained agents.

READ FULL TEXT

page 4

page 6

page 13

page 14

page 15

page 16

page 17

page 18

10/20/2022

Deep reinforcement learning oriented for real world dynamic scenarios

Autonomous navigation in dynamic environments is a complex but essential...
07/06/2020

robo-gym – An Open Source Toolkit for Distributed Deep Reinforcement Learning on Real and Simulated Robots

Applying Deep Reinforcement Learning (DRL) to complex tasks in the field...
11/24/2020

Bi-directional Domain Adaptation for Sim2Real Transfer of Embodied Navigation Agents

Deep reinforcement learning models are notoriously data hungry, yet real...
01/14/2021

Evaluating the Robustness of Collaborative Agents

In order for agents trained by deep reinforcement learning to work along...
08/31/2018

Gibson Env: Real-World Perception for Embodied Agents

Developing visual perception models for active agents and sensorimotor c...
02/11/2020

AI Online Filters to Real World Image Recognition

Deep artificial neural networks, trained with labeled data sets are wide...
07/04/2022

WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents

Existing benchmarks for grounding language in interactive environments e...