
ModelFree Algorithm and Regret Analysis for MDPs with LongTerm Constraints
In the optimization of dynamical systems, the variables typically have c...
read it

A Modelfree Learning Algorithm for Infinitehorizon Averagereward MDPs with Nearoptimal Regret
Recently, modelfree reinforcement learning has attracted research atten...
read it

Modelfree Reinforcement Learning in Infinitehorizon Averagereward Markov Decision Processes
Modelfree reinforcement learning is known to be memory and computation ...
read it

Variance Reduction Methods for Sublinear Reinforcement Learning
This work considers the problem of provably optimal reinforcement learni...
read it

Nearoptimal Bayesian Solution For Unknown Discrete Markov Decision Process
We tackle the problem of acting in an unknown finite and discrete Markov...
read it

Improved Exploration in Factored AverageReward MDPs
We consider a regret minimization task under the averagereward criterio...
read it

ExplorationExploitation in Constrained MDPs
In many sequential decisionmaking problems, the goal is to optimize a u...
read it
ModelFree Algorithm and Regret Analysis for MDPs with Peak Constraints
In the optimization of dynamic systems, the variables typically have constraints. Such problems can be modeled as a constrained Markov Decision Process (MDP). This paper considers a modelfree approach to the problem, where the transition probabilities are not known. In the presence of peak constraints, the agent has to choose the policy to maximize the longterm average reward as well as satisfy the constraints at each time. We propose modifications to the standard Qlearning problem for unconstrained optimization to come up with an algorithm with peak constraints. The proposed algorithm is shown to achieve O(T^1/2+γ) regret bound for the obtained reward, and O(T^1γ) regret bound for the constraint violation for any γ∈(0,1/2) and timehorizon T. We note that these are the first results on regret analysis for constrained MDP, where the transition problems are not known apriori. We demonstrate the proposed algorithm on an energy harvesting problem where it outperforms stateoftheart and performs close to the theoretical upper bound of the studied optimization problem.
READ FULL TEXT
Comments
There are no comments yet.