Consolidated Adaptive T-soft Update for Deep Reinforcement Learning

by   Taisuke Kobayashi, et al.

Demand for deep reinforcement learning (DRL) is gradually increased to enable robots to perform complex tasks, while DRL is known to be unstable. As a technique to stabilize its learning, a target network that slowly and asymptotically matches a main network is widely employed to generate stable pseudo-supervised signals. Recently, T-soft update has been proposed as a noise-robust update rule for the target network and has contributed to improving the DRL performance. However, the noise robustness of T-soft update is specified by a hyperparameter, which should be tuned for each task, and is deteriorated by a simplified implementation. This study develops adaptive T-soft (AT-soft) update by utilizing the update rule in AdaTerm, which has been developed recently. In addition, the concern that the target network does not asymptotically match the main network is mitigated by a new consolidation for bringing the main network back to the target network. This so-called consolidated AT-soft (CAT-soft) update is verified through numerical simulations.


page 1

page 2

page 3

page 4


t-Soft Update of Target Network for Deep Reinforcement Learning

This paper proposes a new robust update rule of the target network for d...

Stable deep reinforcement learning method by predicting uncertainty in rewards as a subtask

In recent years, a variety of tasks have been accomplished by deep reinf...

Deep Reinforcement Learning for Electric Transmission Voltage Control

Today, human operators primarily perform voltage control of the electric...

A Hard and Soft Hybrid Slicing Framework for Service Level Agreement Guarantee via Deep Reinforcement Learning

Network slicing is a critical driver for guaranteeing the diverse servic...

Binarized P-Network: Deep Reinforcement Learning of Robot Control from Raw Images on FPGA

This paper explores a Deep Reinforcement Learning (DRL) approach for des...

Adaptive actuation of magnetic soft robots using deep reinforcement learning

Magnetic soft robots have attracted growing interest due to their unique...

Please sign up or login with your details

Forgot password? Click here to reset