An Introductory Guide to Fano's Inequality with Applications in Statistical Estimation

01/02/2019
by   Jonathan Scarlett, et al.
0

Information theory plays an indispensable role in the development of algorithm-independent impossibility results, both for communication problems and for seemingly distinct areas such as statistics and machine learning. While numerous information-theoretic tools have been proposed for this purpose, the oldest one remains arguably the most versatile and widespread: Fano's inequality. In this chapter, we provide a survey of Fano's inequality and its variants in the context of statistical estimation, adopting a versatile framework that covers a wide range of specific problems. We present a variety of key tools and techniques used for establishing impossibility results via this approach, and provide representative examples covering group testing, graphical model selection, sparse linear regression, density estimation, and convex optimization.

READ FULL TEXT
research
05/21/2020

Chang's lemma via Pinsker's inequality

Extending the idea in [Impagliazzo, R., Moore, C. and Russell, A., An en...
research
09/19/2009

A Bernstein-type inequality for stochastic processes of quadratic forms of Gaussian variables

We introduce a Bernstein-type inequality which serves to uniformly contr...
research
03/09/2022

Cross validation for model selection: a primer with examples from ecology

The growing use of model-selection principles in ecology for statistical...
research
06/14/2017

Strong converse bounds for high-dimensional estimation

In statistical inference problems, we wish to obtain lower bounds on the...
research
11/12/2021

A Reverse Jensen Inequality Result with Application to Mutual Information Estimation

The Jensen inequality is a widely used tool in a multitude of fields, su...
research
08/06/2017

Declarative Statistics

In this work we introduce declarative statistics, a suite of declarative...
research
05/01/2019

On a conditional inequality in Kolmogorov complexity and its applications in communication complexity

Romashchenko and Zimand rom-zim:c:mutualinfo have shown that if we parti...

Please sign up or login with your details

Forgot password? Click here to reset