DeepAI AI Chat
Log In Sign Up

Balancing detectability and performance of attacks on the control channel of Markov Decision Processes

by   Alessio Russo, et al.
KTH Royal Institute of Technology

We investigate the problem of designing optimal stealthy poisoning attacks on the control channel of Markov decision processes (MDPs). This research is motivated by the recent interest of the research community for adversarial and poisoning attacks applied to MDPs, and reinforcement learning (RL) methods. The policies resulting from these methods have been shown to be vulnerable to attacks perturbing the observations of the decision-maker. In such an attack, drawing inspiration from adversarial examples used in supervised learning, the amplitude of the adversarial perturbation is limited according to some norm, with the hope that this constraint will make the attack imperceptible. However, such constraints do not grant any level of undetectability and do not take into account the dynamic nature of the underlying Markov process. In this paper, we propose a new attack formulation, based on information-theoretical quantities, that considers the objective of minimizing the detectability of the attack as well as the performance of the controlled process. We analyze the trade-off between the efficiency of the attack and its detectability. We conclude with examples and numerical simulations illustrating this trade-off.


page 1

page 2

page 3

page 4


An Overview for Markov Decision Processes in Queues and Networks

Markov decision processes (MDPs) in queues and networks have been an int...

A Tensor Network Approach to Finite Markov Decision Processes

Tensor network (TN) techniques - often used in the context of quantum ma...

Optimizing Quantiles in Preference-based Markov Decision Processes

In the Markov decision process model, policies are usually evaluated by ...

Optimal Attacks on Reinforcement Learning Policies

Control policies, trained using the Deep Reinforcement Learning, have be...

Lecture Notes on Partially Known MDPs

In these notes we will tackle the problem of finding optimal policies fo...

Fairness with Dynamics

It has recently been shown that if feedback effects of decisions are ign...

On Incorporating Forecasts into Linear State Space Model Markov Decision Processes

Weather forecast information will very likely find increasing applicatio...