Dualizing Le Cam's method, with applications to estimating the unseens

02/14/2019
by   Yury Polyanskiy, et al.
0

One of the most commonly used techniques for proving statistical lower bounds, Le Cam's method, has been the method of choice for functional estimation. This papers aims at explaining the effectiveness of Le Cam's method from an optimization perspective. Under a variety of settings it is shown that the maximization problem that searches for the best lower bound provided by Le Cam's method, upon dualizing, becomes a minimization problem that optimizes the bias-variance tradeoff among a family of estimators. For estimating linear functionals of a distribution our work strengthens prior results of Dohono-Liu DL91 (for quadratic loss) by dropping the Hölderian assumption on the modulus of continuity. For exponential families our results improve those of Juditsky-Nemirovski JN09 by characterizing the minimax risk for the quadratic loss under weaker assumptions on the exponential family. We also provide an extension to the high-dimensional setting for estimating separable functionals and apply it to obtain sharp rates in the general area of "estimating the unseens": 1. Distinct elements problem: Randomly sampling a fraction p of colored balls from an urn containing d balls in total, the optimal normalized estimation error of the number of distinct colors in the urn is within logarithmic factors of d^-1/2{p/1-p,1}, exhibiting an elbow at p=1/2. 2. Fisher's species problem: Given n independent samples drawn from an unknown distribution, the optimal normalized prediction error of the number of unseen symbols in the next (unobserved) r · n samples is within logarithmic factors of n^-{1/r+1,1/2}, exhibiting an elbow at r=1.

READ FULL TEXT
research
05/17/2022

New Lower Bounds for Private Estimation and a Generalized Fingerprinting Lemma

We prove new lower bounds for statistical estimation tasks under the con...
research
05/19/2013

Horizon-Independent Optimal Prediction with Log-Loss in Exponential Families

We study online learning under logarithmic loss with regular parametric ...
research
04/27/2020

Minimax testing and quadratic functional estimation for circular convolution

In a circular convolution model, we aim to infer on the density of a cir...
research
04/05/2019

Parameter estimation for integer-valued Gibbs distributions

We consider the family of Gibbs distributions, which are probability dis...
research
01/12/2022

On the Statistical Complexity of Sample Amplification

Given n i.i.d. samples drawn from an unknown distribution P, when is it ...
research
06/25/2015

Analyzing statistical and computational tradeoffs of estimation procedures

The recent explosion in the amount and dimensionality of data has exacer...
research
07/12/2017

Estimating the unseen from multiple populations

Given samples from a distribution, how many new elements should we expec...

Please sign up or login with your details

Forgot password? Click here to reset