
Lowrank Stateaction Valuefunction Approximation
Value functions are central to Dynamic Programming and Reinforcement Lea...
read it

Learning NonParametric Basis Independent Models from Point Queries via LowRank Methods
We consider the problem of learning multiridge functions of the form f(...
read it

A Universal Variance ReductionBased Catalyst for Nonconvex LowRank Matrix Recovery
We propose a generic framework based on a new stochastic variancereduce...
read it

Unifying Framework for Crowdsourcing via Graphon Estimation
We consider the question of inferring true answers associated with tasks...
read it

Incremental Truncated LSTD
Balancing between computational efficiency and sample efficiency is an i...
read it

Fast and SampleEfficient Federated Low Rank Matrix Recovery from Columnwise Linear and Quadratic Projections
This work studies the following problem and its magnitudeonly extension...
read it

On the computational and statistical complexity of overparameterized matrix sensing
We consider solving the low rank matrix sensing problem with Factorized ...
read it
Sample Efficient Reinforcement Learning via LowRank Matrix Estimation
We consider the question of learning Qfunction in a sample efficient manner for reinforcement learning with continuous state and action spaces under a generative model. If Qfunction is Lipschitz continuous, then the minimal sample complexity for estimating ϵoptimal Qfunction is known to scale as Ω(1/ϵ^d_1+d_2 +2) per classical nonparametric learning theory, where d_1 and d_2 denote the dimensions of the state and action spaces respectively. The Qfunction, when viewed as a kernel, induces a HilbertSchmidt operator and hence possesses squaresummable spectrum. This motivates us to consider a parametric class of Qfunctions parameterized by its "rank" r, which contains all Lipschitz Qfunctions as r →∞. As our key contribution, we develop a simple, iterative learning algorithm that finds ϵoptimal Qfunction with sample complexity of O(1/ϵ^max(d_1, d_2)+2) when the optimal Qfunction has low rank r and the discounting factor γ is below a certain threshold. Thus, this provides an exponential improvement in sample complexity. To enable our result, we develop a novel Matrix Estimation algorithm that faithfully estimates an unknown lowrank matrix in the ℓ_∞ sense even in the presence of arbitrary bounded noise, which might be of interest in its own right. Empirical results on several stochastic control tasks confirm the efficacy of our "lowrank" algorithms.
READ FULL TEXT
Comments
There are no comments yet.