DREAM Architecture: a Developmental Approach to Open-Ended Learning in Robotics

05/13/2020
by   Stéphane Doncieux, et al.
22

Robots are still limited to controlled conditions, that the robot designer knows with enough details to endow the robot with the appropriate models or behaviors. Learning algorithms add some flexibility with the ability to discover the appropriate behavior given either some demonstrations or a reward to guide its exploration with a reinforcement learning algorithm. Reinforcement learning algorithms rely on the definition of state and action spaces that define reachable behaviors. Their adaptation capability critically depends on the representations of these spaces: small and discrete spaces result in fast learning while large and continuous spaces are challenging and either require a long training period or prevent the robot from converging to an appropriate behavior. Beside the operational cycle of policy execution and the learning cycle, which works at a slower time scale to acquire new policies, we introduce the redescription cycle, a third cycle working at an even slower time scale to generate or adapt the required representations to the robot, its environment and the task. We introduce the challenges raised by this cycle and we present DREAM (Deferred Restructuring of Experience in Autonomous Machines), a developmental cognitive architecture to bootstrap this redescription process stage by stage, build new state representations with appropriate motivations, and transfer the acquired knowledge across domains or tasks or even across robots. We describe results obtained so far with this approach and end up with a discussion of the questions it raises in Neuroscience.

READ FULL TEXT

page 1

page 8

page 9

page 11

page 12

page 14

page 16

page 18

research
03/02/2018

Unsupervised Learning of Goal Spaces for Intrinsically Motivated Goal Exploration

Intrinsically motivated goal exploration algorithms enable machines to d...
research
09/24/2022

Fast Lifelong Adaptive Inverse Reinforcement Learning from Demonstrations

Learning from Demonstration (LfD) approaches empower end-users to teach ...
research
10/17/2017

Map-based Multi-Policy Reinforcement Learning: Enhancing Adaptability of Robots by Deep Reinforcement Learning

In order for robots to perform mission-critical tasks, it is essential t...
research
08/09/2022

On the Importance of Critical Period in Multi-stage Reinforcement Learning

The initial years of an infant's life are known as the critical period, ...
research
08/30/2020

Human-in-the-Loop Methods for Data-Driven and Reinforcement Learning Systems

Recent successes combine reinforcement learning algorithms and deep neur...
research
03/28/2013

Design for a Darwinian Brain: Part 2. Cognitive Architecture

The accumulation of adaptations in an open-ended manner during lifetime ...
research
11/07/2018

Generative Adversarial Policy Networks for Behavioural Repertoire

Learning algorithms are enabling robots to solve increasingly challengin...

Please sign up or login with your details

Forgot password? Click here to reset