DeepAI AI Chat
Log In Sign Up

Action Robust Reinforcement Learning and Applications in Continuous Control

by   Chen Tessler, et al.

A policy is said to be robust if it maximizes the reward while considering a bad, or even adversarial, model. In this work we formalize two new criteria of robustness to action uncertainty. Specifically, we consider two scenarios in which the agent attempts to perform an action a, and (i) with probability α, an alternative adversarial action a̅ is taken, or (ii) an adversary adds a perturbation to the selected action in the case of continuous action space. We show that our criteria are related to common forms of uncertainty in robotics domains, such as the occurrence of abrupt forces, and suggest algorithms in the tabular case. Building on the suggested algorithms, we generalize our approach to deep reinforcement learning (DRL) and provide extensive experiments in the various MuJoCo domains. Our experiments show that not only does our approach produce robust policies, but it also improves the performance in the absence of perturbations. This generalization indicates that action-robustness can be thought of as implicit regularization in RL problems.


page 7

page 25

page 26


Robust Deep Reinforcement Learning against Adversarial Perturbations on Observations

Deep Reinforcement Learning (DRL) is vulnerable to small adversarial per...

What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?

Various methods for Multi-Agent Reinforcement Learning (MARL) have been ...

Falsification-Based Robust Adversarial Reinforcement Learning

Reinforcement learning (RL) has achieved tremendous progress in solving ...

CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothing

We present the first framework of Certifying Robust Policies for reinfor...

Robust Adversarial Reinforcement Learning

Deep neural networks coupled with fast simulation and improved computati...

Certified Adversarial Robustness for Deep Reinforcement Learning

Deep Neural Network-based systems are now the state-of-the-art in many r...

Characterising the Robustness of Reinforcement Learning for Continuous Control using Disturbance Injection

In this study, we leverage the deliberate and systematic fault-injection...