DeepAI
Log In Sign Up

Structured Logconcave Sampling with a Restricted Gaussian Oracle

10/07/2020
by   Yin Tat Lee, et al.
0

We give algorithms for sampling several structured logconcave families to high accuracy. We further develop a reduction framework, inspired by proximal point methods in convex optimization, which bootstraps samplers for regularized densities to improve dependences on problem conditioning. A key ingredient in our framework is the notion of a "restricted Gaussian oracle" (RGO) for g: ℝ^d →ℝ, which is a sampler for distributions whose negative log-likelihood sums a quadratic and g. By combining our reduction framework with our new samplers, we obtain the following bounds for sampling structured distributions to total variation distance ϵ. For composite densities (-f(x) - g(x)), where f has condition number κ and convex (but possibly non-smooth) g admits an RGO, we obtain a mixing time of O(κ d log^3κ d/ϵ), matching the state-of-the-art non-composite bound; no composite samplers with better mixing than general-purpose logconcave samplers were previously known. For logconcave finite sums (-F(x)), where F(x) = 1/n∑_i ∈ [n] f_i(x) has condition number κ, we give a sampler querying O(n + κmax(d, √(nd))) gradient oracles to {f_i}_i ∈ [n]; no high-accuracy samplers with nontrivial gradient query complexity were previously known. For densities with condition number κ, we give an algorithm obtaining mixing time O(κ d log^2κ d/ϵ), improving the prior state-of-the-art by a logarithmic factor with a significantly simpler analysis; we also show a zeroth-order algorithm attains the same query complexity.

READ FULL TEXT

page 1

page 2

page 3

page 4

06/10/2020

Composite Logconcave Sampling with a Restricted Gaussian Oracle

We consider sampling from composite densities on ℝ^d of the form dπ(x) ∝...
02/13/2022

Improved analysis for a proximal algorithm for sampling

We study the proximal sampler of Lee, Shen, and Tian (2021) and obtain n...
10/01/2019

An Efficient Sampling Algorithm for Non-smooth Composite Potentials

We consider the problem of sampling from a density of the form p(x) ∝(-f...
10/09/2021

A Proximal Algorithm for Sampling from Non-smooth Potentials

Markov chain Monte Carlo (MCMC) is an effective and dominant method to s...
02/28/2022

A Proximal Algorithm for Sampling

We consider sampling problems with possibly non-smooth potentials (negat...
06/17/2022

RECAPP: Crafting a More Efficient Catalyst for Convex Optimization

The accelerated proximal point algorithm (APPA), also known as "Catalyst...
01/24/2014

The Sampling-and-Learning Framework: A Statistical View of Evolutionary Algorithms

Evolutionary algorithms (EAs), a large class of general purpose optimiza...