
Nearly Optimal Regret for Learning Adversarial MDPs with Linear Function Approximation
We study the reinforcement learning for finitehorizon episodic Markov d...
read it

Nearly Minimax Optimal Regret for Learning Infinitehorizon Averagereward MDPs with Linear Function Approximation
We study reinforcement learning in an infinitehorizon averagereward se...
read it

Logarithmic Regret for Reinforcement Learning with Linear Function Approximation
Reinforcement learning (RL) with linear function approximation has recei...
read it

Online learning in MDPs with side information
We study online learning of finite Markov decision process (MDP) problem...
read it

Solving Discounted Stochastic TwoPlayer Games with NearOptimal Time and Sample Complexity
In this paper, we settle the sampling complexity of solving discounted t...
read it

Provably Breaking the Quadratic Error Compounding Barrier in Imitation Learning, Optimally
We study the statistical limits of Imitation Learning (IL) in episodic M...
read it

RL for Latent MDPs: Regret Guarantees and a Lower Bound
In this work, we consider the regret minimization problem for reinforcem...
read it
Nearly Minimax Optimal Reinforcement Learning for Linear Mixture Markov Decision Processes
We study reinforcement learning (RL) with linear function approximation where the underlying transition probability kernel of the Markov decision process (MDP) is a linear mixture model (Jia et al., 2020; Ayoub et al., 2020; Zhou et al., 2020) and the learning agent has access to either an integration or a sampling oracle of the individual basis kernels. We propose a new Bernsteintype concentration inequality for selfnormalized martingales for linear bandit problems with bounded noise. Based on the new inequality, we propose a new, computationally efficient algorithm with linear function approximation named UCRLVTR^+ for the aforementioned linear mixture MDPs in the episodic undiscounted setting. We show that UCRLVTR^+ attains an Õ(dH√(T)) regret where d is the dimension of feature mapping, H is the length of the episode and T is the number of interactions with the MDP. We also prove a matching lower bound Ω(dH√(T)) for this setting, which shows that UCRLVTR^+ is minimax optimal up to logarithmic factors. In addition, we propose the UCLK^+ algorithm for the same family of MDPs under discounting and show that it attains an Õ(d√(T)/(1γ)^1.5) regret, where γ∈ [0,1) is the discount factor. Our upper bound matches the lower bound Ω(d√(T)/(1γ)^1.5) proved by Zhou et al. (2020) up to logarithmic factors, suggesting that UCLK^+ is nearly minimax optimal. To the best of our knowledge, these are the first computationally efficient, nearly minimax optimal algorithms for RL with linear function approximation.
READ FULL TEXT
Comments
There are no comments yet.