Log In Sign Up

Model-Based Quality-Diversity Search for Efficient Robot Learning

by   Leon Keller, et al.

Despite recent progress in robot learning, it still remains a challenge to program a robot to deal with open-ended object manipulation tasks. One approach that was recently used to autonomously generate a repertoire of diverse skills is a novelty based Quality-Diversity (QD) algorithm. However, as most evolutionary algorithms, QD suffers from sample-inefficiency and, thus, it is challenging to apply it in real-world scenarios. This paper tackles this problem by integrating a neural network that predicts the behavior of the perturbed parameters into a novelty based QD algorithm. In the proposed Model-based Quality-Diversity search (M-QD), the network is trained concurrently to the repertoire and is used to avoid executing unpromising actions in the novelty search process. Furthermore, it is used to adapt the skills of the final repertoire in order to generalize the skills to different scenarios. Our experiments show that enhancing a QD algorithm with such a forward model improves the sample-efficiency and performance of the evolutionary process and the skill adaptation.


page 1

page 4


Online Damage Recovery for Physical Robots with Hierarchical Quality-Diversity

In real-world environments, robots need to be resilient to damages and r...

From exploration to control: learning object manipulation skills through novelty search and local adaptation

Programming a robot to deal with open-ended tasks remains a challenge, i...

Emergence of Novelty in Evolutionary Algorithms

One of the main problems of evolutionary algorithms is the convergence o...

Diversity and Novelty MasterPrints: Generating Multiple DeepMasterPrints for Increased User Coverage

This work expands on previous advancements in genetic fingerprint spoofi...

Representation Edit Distance as a Measure of Novelty

Adaptation to novelty is viewed as learning to change and augment existi...

Hierarchical Quality-Diversity for Online Damage Recovery

Adaptation capabilities, like damage recovery, are crucial for the deplo...