Dualizing Le Cam's method, with applications to estimating the unseens

by   Yury Polyanskiy, et al.

One of the most commonly used techniques for proving statistical lower bounds, Le Cam's method, has been the method of choice for functional estimation. This papers aims at explaining the effectiveness of Le Cam's method from an optimization perspective. Under a variety of settings it is shown that the maximization problem that searches for the best lower bound provided by Le Cam's method, upon dualizing, becomes a minimization problem that optimizes the bias-variance tradeoff among a family of estimators. For estimating linear functionals of a distribution our work strengthens prior results of Dohono-Liu DL91 (for quadratic loss) by dropping the Hölderian assumption on the modulus of continuity. For exponential families our results improve those of Juditsky-Nemirovski JN09 by characterizing the minimax risk for the quadratic loss under weaker assumptions on the exponential family. We also provide an extension to the high-dimensional setting for estimating separable functionals and apply it to obtain sharp rates in the general area of "estimating the unseens": 1. Distinct elements problem: Randomly sampling a fraction p of colored balls from an urn containing d balls in total, the optimal normalized estimation error of the number of distinct colors in the urn is within logarithmic factors of d^-1/2{p/1-p,1}, exhibiting an elbow at p=1/2. 2. Fisher's species problem: Given n independent samples drawn from an unknown distribution, the optimal normalized prediction error of the number of unseen symbols in the next (unobserved) r · n samples is within logarithmic factors of n^-{1/r+1,1/2}, exhibiting an elbow at r=1.


New Lower Bounds for Private Estimation and a Generalized Fingerprinting Lemma

We prove new lower bounds for statistical estimation tasks under the con...

Horizon-Independent Optimal Prediction with Log-Loss in Exponential Families

We study online learning under logarithmic loss with regular parametric ...

Minimax testing and quadratic functional estimation for circular convolution

In a circular convolution model, we aim to infer on the density of a cir...

Parameter estimation for integer-valued Gibbs distributions

We consider the family of Gibbs distributions, which are probability dis...

On the Statistical Complexity of Sample Amplification

Given n i.i.d. samples drawn from an unknown distribution P, when is it ...

Analyzing statistical and computational tradeoffs of estimation procedures

The recent explosion in the amount and dimensionality of data has exacer...

Estimating the unseen from multiple populations

Given samples from a distribution, how many new elements should we expec...

Please sign up or login with your details

Forgot password? Click here to reset