Efficient Truncated Statistics with Unknown Truncation

08/02/2019
by   Vasilis Kontonis, et al.
0

We study the problem of estimating the parameters of a Gaussian distribution when samples are only shown if they fall in some (unknown) subset S ⊆^d. This core problem in truncated statistics has long history going back to Galton, Lee, Pearson and Fisher. Recent work by Daskalakis et al. (FOCS'18), provides the first efficient algorithm that works for arbitrary sets in high dimension when the set is known, but leaves as an open problem the more challenging and relevant case of unknown truncation set. Our main result is a computationally and sample efficient algorithm for estimating the parameters of the Gaussian under arbitrary unknown truncation sets whose performance decays with a natural measure of complexity of the set, namely its Gaussian surface area. Notably, this algorithm works for large families of sets including intersections of halfspaces, polynomial threshold functions and general convex sets. We show that our algorithm closely captures the tradeoff between the complexity of the set and the number of samples needed to learn the parameters by exhibiting a set with small Gaussian surface area for which it is information theoretically impossible to learn the true Gaussian with few samples.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/11/2018

Efficient Statistics, in High Dimensions, from Truncated Samples

We provide an efficient algorithm for the classical problem, going back ...
research
05/04/2023

Testing Convex Truncation

We study the basic statistical problem of testing whether normally distr...
research
06/17/2020

Efficient Statistics for Sparse Graphical Models from Truncated Samples

In this paper, we study high-dimensional estimation from truncated sampl...
research
11/03/2019

Testing noisy linear functions for sparsity

We consider the following basic inference problem: there is an unknown h...
research
07/05/2020

Efficient Parameter Estimation of Truncated Boolean Product Distributions

We study the problem of estimating the parameters of a Boolean product d...
research
06/16/2020

Reverse Lebesgue and Gaussian isoperimetric inequalities for parallel sets with applications

The r-parallel set of a measurable set A ⊆ℝ^d is the set of all points w...
research
03/25/2023

Exact Short Products From Truncated Multipliers

We sometimes need to compute the most significant digits of the product ...

Please sign up or login with your details

Forgot password? Click here to reset