Log In Sign Up

Privacy-Preserving Policy Synthesis in Markov Decision Processes

by   Parham Gohari, et al.

In decision-making problems, the actions of an agent may reveal sensitive information that drives its decisions. For instance, a corporation's investment decisions may reveal its sensitive knowledge about market dynamics. To prevent this type of information leakage, we introduce a policy synthesis algorithm that protects the privacy of the transition probabilities in a Markov decision process. We use differential privacy as the mathematical definition of privacy. The algorithm first perturbs the transition probabilities using a mechanism that provides differential privacy. Then, based on the privatized transition probabilities, we synthesize a policy using dynamic programming. Our main contribution is to bound the "cost of privacy," i.e., the difference between the expected total rewards with privacy and the expected total rewards without privacy. We also show that computing the cost of privacy has time complexity that is polynomial in the parameters of the problem. Moreover, we establish that the cost of privacy increases with the strength of differential privacy protections, and we quantify this increase. Finally, numerical experiments on two example environments validate the established relationship between the cost of privacy and the strength of data privacy protections.


page 1

page 2

page 3

page 4


The Dirichlet Mechanism for Differential Privacy on the Unit Simplex

As members of a network share more information with each other and netwo...

Differential Privacy in Cooperative Multiagent Planning

Privacy-aware multiagent systems must protect agents' sensitive data whi...

Offline Reinforcement Learning with Differential Privacy

The offline reinforcement learning (RL) problem is often motivated by th...

The Smoothed Complexity of Policy Iteration for Markov Decision Processes

We show subexponential lower bounds (i.e., 2^Ω (n^c)) on the smoothed co...

Towards Differential Privacy for Symbolic Systems

In this paper, we develop a privacy implementation for symbolic control ...

Correspondences between Privacy and Nondiscrimination: Why They Should Be Studied Together

Privacy and nondiscrimination are related but different. We make this ob...

Privacy Risk for anisotropic Langevin dynamics using relative entropy bounds

The privacy preserving properties of Langevin dynamics with additive iso...