DeepAI AI Chat
Log In Sign Up

C^2:Co-design of Robots via Concurrent Networks Coupling Online and Offline Reinforcement Learning

09/14/2022
by   Ci Chen, et al.
Zhejiang University
0

With the rise of computing power, using data-driven approaches for co-designing robots' morphology and controller has become a feasible way. Nevertheless, evaluating the fitness of the controller under each morphology is time-consuming. As a pioneering data-driven method, Co-adaptation utilizes a double-network mechanism with the aim of learning a Q function conditioned on morphology parameters to replace the traditional evaluation of a diverse set of candidates, thereby speeding up optimization. In this paper, we find that Co-adaptation ignores the existence of exploration error during training and state-action distribution shift during parameter transmitting, which hurt the performance. We propose the framework of the concurrent network that couples online and offline RL methods. By leveraging the behavior cloning term flexibly, we mitigate the impact of the above issues on the results. Simulation and physical experiments are performed to demonstrate that our proposed method outperforms baseline algorithms, which illustrates that the proposed method is an effective way of discovering the optimal combination of morphology and controller.

READ FULL TEXT
11/15/2019

Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learning

Humans and animals are capable of quickly learning new behaviours to sol...
06/19/2021

Boosting Offline Reinforcement Learning with Residual Generative Modeling

Offline reinforcement learning (RL) tries to learn the near-optimal poli...
06/17/2022

AnyMorph: Learning Transferable Polices By Inferring Agent Morphology

The prototypical approach to reinforcement learning involves training po...
02/22/2023

Universal Morphology Control via Contextual Modulation

Learning a universal policy across different robot morphologies can sign...
10/20/2021

Data-Driven Offline Optimization For Architecting Hardware Accelerators

Industry has gradually moved towards application-specific hardware accel...
05/03/2019

Data-efficient Learning of Morphology and Controller for a Microrobot

Robot design is often a slow and difficult process requiring the iterati...
11/24/2022

Control and Morphology Optimization of Passive Asymmetric Structures for Robotic Swimming

Aquatic creatures exhibit remarkable adaptations of their body to effici...