Parameter estimation for integer-valued Gibbs distributions

04/05/2019
by   David G. Harris, et al.
0

We consider the family of Gibbs distributions, which are probability distributions over a discrete space Ω given by μ^Ω_β(x)=e^β H(x)/Z(β). Here H:Ω→{0,1,...,n} is a fixed function (called a Hamiltonian), β is the parameter of the distribution, and Z(β)=∑_x∈Ωe^β H(x) = ∑_k=0^n c_k e^β k is the normalization factor called the partition function. We study how function Z(·) can be estimated using an oracle that produces samples x∼μ_β(·) for a value β in a given interval [β_, β_]. Recently, it has been shown how to estimate quantity q=Z(β_)/Z(β_) with additive error ε using Õ(q/ε^2) samples in expectation. We improve this result to Õ({q,n^2}/ε^2), matching a lower bound of Kolmogorov (2018) up to logarithmic factors. We also consider the problem of estimating the normalized coefficients c_k for indices k∈{0,1,...,n} that satisfy _βμ_β^Ω({x | H(x)=k}) >μ_∗, where μ_∗∈(0,1) is a given parameter. We solve this problem using Õ({ q + √(q)/μ_∗, n^2 + n/μ_∗}/ε^2) expected samples, and we show that this complexity is optimal up to logarithmic factors. This is improved to roughly Õ( 1/μ_∗+{q + n,n^2}/ε^2) for applications in which the coefficients are known to be log-concave (e.g. for connected subgraphs of a given graph).

READ FULL TEXT
07/17/2020

Parameter estimation for Gibbs distributions

We consider Gibbs distributions, which are families of probability distr...
11/09/2020

Characterizations of non-normalized discrete probability distributions and their application in statistics

From the distributional characterizations that lie at the heart of Stein...
02/14/2019

Dualizing Le Cam's method, with applications to estimating the unseens

One of the most commonly used techniques for proving statistical lower b...
04/15/2020

Sample-efficient learning of quantum many-body systems

We study the problem of learning the Hamiltonian of a quantum many-body ...
01/02/2018

A Concentration Result of Estimating Phi-Divergence using Data Dependent Partition

Estimation of the ϕ-divergence between two unknown probability distribut...
11/02/2019

Order Optimal One-Shot Distributed Learning

We consider distributed statistical optimization in one-shot setting, wh...
06/19/2022

Deterministic Finite-Memory Bias Estimation

In this paper we consider the problem of estimating a Bernoulli paramete...