Causal Inference Q-Network: Toward Resilient Reinforcement Learning

by   Chao-Han Huck Yang, et al.

Deep reinforcement learning (DRL) has demonstrated impressive performance in various gaming simulators and real-world applications. In practice, however, a DRL agent may receive faulty observation by abrupt interferences such as black-out, frozen-screen, and adversarial perturbation. How to design a resilient DRL algorithm against these rare but mission-critical and safety-crucial scenarios is an important yet challenging task. In this paper, we consider a generative DRL framework training with an auxiliary task of observational interferences such as artificial noises. Under this framework, we discuss the importance of the causal relation and propose a causal inference based DRL algorithm called causal inference Q-network (CIQ). We evaluate the performance of CIQ in several benchmark DRL environments with different types of interferences as auxiliary labels. Our experimental results show that the proposed CIQ method could achieve higher performance and more resilience against observational interferences.



There are no comments yet.


page 11

page 14

page 15

page 17

page 18

page 20

page 21

page 23


Pessimistic Model Selection for Offline Deep Reinforcement Learning

Deep Reinforcement Learning (DRL) has demonstrated great potentials in s...

DDPG car-following model with real-world human driving experience in CARLA

In the autonomous driving field, the fusion of human knowledge into Deep...

Dependability Analysis of Deep Reinforcement Learning based Robotics and Autonomous Systems

While Deep Reinforcement Learning (DRL) provides transformational capabi...

Enhancing the Generalization Performance and Speed Up Training for DRL-based Mapless Navigation

Training an agent to navigate with DRL is data-hungry, which requires mi...

Provably Efficient Causal Reinforcement Learning with Confounded Observational Data

Empowered by expressive function approximators such as neural networks, ...

Hypernetwork Dismantling via Deep Reinforcement Learning

Network dismantling aims to degrade the connectivity of a network by rem...

Deep Echo State Q-Network (DEQN) and Its Application in Dynamic Spectrum Sharing for 5G and Beyond

Deep reinforcement learning (DRL) has been shown to be successful in man...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.