Dualizing Le Cam's method, with applications to estimating the unseens
share
One of the most commonly used techniques for proving statistical lower bounds, Le Cam's method, has been the method of choice for functional estimation. This papers aims at explaining the effectiveness of Le Cam's method from an optimization perspective. Under a variety of settings it is shown that the maximization problem that searches for the best lower bound provided by Le Cam's method, upon dualizing, becomes a minimization problem that optimizes the bias-variance tradeoff among a family of estimators. For estimating linear functionals of a distribution our work strengthens prior results of Dohono-Liu DL91 (for quadratic loss) by dropping the Hölderian assumption on the modulus of continuity. For exponential families our results improve those of Juditsky-Nemirovski JN09 by characterizing the minimax risk for the quadratic loss under weaker assumptions on the exponential family. We also provide an extension to the high-dimensional setting for estimating separable functionals and apply it to obtain sharp rates in the general area of "estimating the unseens": 1. Distinct elements problem: Randomly sampling a fraction p of colored balls from an urn containing d balls in total, the optimal normalized estimation error of the number of distinct colors in the urn is within logarithmic factors of d^-1/2{p/1-p,1}, exhibiting an elbow at p=1/2. 2. Fisher's species problem: Given n independent samples drawn from an unknown distribution, the optimal normalized prediction error of the number of unseen symbols in the next (unobserved) r · n samples is within logarithmic factors of n^-{1/r+1,1/2}, exhibiting an elbow at r=1.
READ FULL TEXT