Generalization of Reinforcement Learning with Policy-Aware Adversarial Data Augmentation

by   Hanping Zhang, et al.

The generalization gap in reinforcement learning (RL) has been a significant obstacle that prevents the RL agent from learning general skills and adapting to varying environments. Increasing the generalization capacity of the RL systems can significantly improve their performance on real-world working environments. In this work, we propose a novel policy-aware adversarial data augmentation method to augment the standard policy learning method with automatically generated trajectory data. Different from the commonly used observation transformation based data augmentations, our proposed method adversarially generates new trajectory data based on the policy gradient objective and aims to more effectively increase the RL agent's generalization ability with the policy-aware data augmentation. Moreover, we further deploy a mixup step to integrate the original and generated data to enhance the generalization capacity while mitigating the over-deviation of the adversarial data. We conduct experiments on a number of RL tasks to investigate the generalization performance of the proposed method by comparing it with the standard baselines and the state-of-the-art mixreg approach. The results show our method can generalize well with limited training diversity, and achieve the state-of-the-art generalization test performance.



There are no comments yet.


page 1

page 2

page 3

page 4


Automatic Data Augmentation for Generalization in Deep Reinforcement Learning

Deep reinforcement learning (RL) agents often fail to generalize to unse...

Improving Generalization in Reinforcement Learning with Mixture Regularization

Deep reinforcement learning (RL) agents trained in a limited set of envi...

A Graph Attention Learning Approach to Antenna Tilt Optimization

6G will move mobile networks towards increasing levels of complexity. To...

Adversarial Reinforcement Learning for Procedural Content Generation

We present an approach for procedural content generation (PCG), and impr...

Learning Data Manipulation for Augmentation and Weighting

Manipulating data, such as weighting data examples or augmenting with ne...

Robust Reinforcement Learning via Adversarial training with Langevin Dynamics

We introduce a sampling perspective to tackle the challenging task of tr...

Value Driven Representation for Human-in-the-Loop Reinforcement Learning

Interactive adaptive systems powered by Reinforcement Learning (RL) have...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.