Efficient Algorithms for Planning with Participation Constraints

05/16/2022
by   Hanrui Zhang, et al.
0

We consider the problem of planning with participation constraints introduced in [Zhang et al., 2022]. In this problem, a principal chooses actions in a Markov decision process, resulting in separate utilities for the principal and the agent. However, the agent can and will choose to end the process whenever his expected onward utility becomes negative. The principal seeks to compute and commit to a policy that maximizes her expected utility, under the constraint that the agent should always want to continue participating. We provide the first polynomial-time exact algorithm for this problem for finite-horizon settings, where previously only an additive ε-approximation algorithm was known. Our approach can also be extended to the (discounted) infinite-horizon case, for which we give an algorithm that runs in time polynomial in the size of the input and log(1/ε), and returns a policy that is optimal up to an additive error of ε.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/14/2023

On Supermodular Contracts and Dense Subgraphs

We study the combinatorial contract design problem, introduced and studi...
research
05/13/2021

Automated Dynamic Mechanism Design

We study Bayesian automated mechanism design in unstructured dynamic env...
research
01/29/2020

Asymptotically Efficient Off-Policy Evaluation for Tabular Reinforcement Learning

We consider the problem of off-policy evaluation for reinforcement learn...
research
10/14/2021

The Geometry of Memoryless Stochastic Policy Optimization in Infinite-Horizon POMDPs

We consider the problem of finding the best memoryless stochastic policy...
research
03/24/2022

Horizon-Free Reinforcement Learning in Polynomial Time: the Power of Stationary Policies

This paper gives the first polynomial-time algorithm for tabular Markov ...
research
11/14/2019

Deception through Half-Truths

Deception is a fundamental issue across a diverse array of settings, fro...
research
09/12/2021

Concave Utility Reinforcement Learning with Zero-Constraint Violations

We consider the problem of tabular infinite horizon concave utility rein...

Please sign up or login with your details

Forgot password? Click here to reset