An exact solution in Markov decision process with multiplicative rewards as a general framework

12/15/2020
by   Yuan Yao, et al.
0

We develop an exactly solvable framework of Markov decision process with a finite horizon, and continuous state and action spaces. We first review the exact solution of conventional linear quadratic regulation with a linear transition and a Gaussian noise, whose optimal policy does not depend on the Gaussian noise, which is an undesired feature in the presence of significant noises. It motivates us to investigate exact solutions which depend on noise. To do so, we generalize the reward accumulation to be a general binary commutative and associative operation. By a new multiplicative accumulation, we obtain an exact solution of optimization assuming linear transitions with a Gaussian noise and the optimal policy is noise dependent in contrast to the additive accumulation. Furthermore, we also show that the multiplicative scheme is a general framework that covers the additive one with an arbitrary precision, which is a model-independent principle.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/17/2017

Primal-Dual π Learning: Sample Complexity and Sublinear Run Time for Ergodic Markov Decision Problems

Consider the problem of approximating the optimal policy of a Markov dec...
research
01/16/2020

Weak Detection in the Spiked Wigner Model with General Rank

We study the statistical decision process of detecting the presence of s...
research
11/05/2019

A Note on Quantum Markov Models

The study of Markov models is central to control theory and machine lear...
research
05/17/2018

Memoryless Exact Solutions for Deterministic MDPs with Sparse Rewards

We propose an algorithm for deterministic continuous Markov Decision Pro...
research
03/17/2021

Impact of the error structure on the design and analysis of enzyme kinetic models

The statistical analysis of enzyme kinetic reactions usually involves mo...
research
03/28/2023

Concentration of Contractive Stochastic Approximation: Additive and Multiplicative Noise

In this work, we study the concentration behavior of a stochastic approx...
research
05/17/2012

Optimal Weights Mixed Filter for Removing Mixture of Gaussian and Impulse Noises

According to the character of Gaussian, we modify the Rank-Ordered Absol...

Please sign up or login with your details

Forgot password? Click here to reset