Pointwise Bounds for Distribution Estimation under Communication Constraints

10/07/2021
by   Wei-Ning Chen, et al.
0

We consider the problem of estimating a d-dimensional discrete distribution from its samples observed under a b-bit communication constraint. In contrast to most previous results that largely focus on the global minimax error, we study the local behavior of the estimation error and provide pointwise bounds that depend on the target distribution p. In particular, we show that the ℓ_2 error decays with O(‖ p‖_1/2/n2^b∨1/n) (In this paper, we use a∨ b and a ∧ b to denote max(a, b) and min(a,b) respectively.) when n is sufficiently large, hence it is governed by the half-norm of p instead of the ambient dimension d. For the achievability result, we propose a two-round sequentially interactive estimation scheme that achieves this error rate uniformly over all p. Our scheme is based on a novel local refinement idea, where we first use a standard global minimax scheme to localize p and then use the remaining samples to locally refine our estimate. We also develop a new local minimax lower bound with (almost) matching ℓ_2 error, showing that any interactive scheme must admit a Ω( ‖ p ‖_(1+δ)/2/n2^b) ℓ_2 error for any δ > 0. The lower bound is derived by first finding the best parametric sub-model containing p, and then upper bounding the quantized Fisher information under this model. Our upper and lower bounds together indicate that the ℋ_1/2(p) = log(‖ p ‖_1/2) bits of communication is both sufficient and necessary to achieve the optimal (centralized) performance, where ℋ_1/2(p) is the Rényi entropy of order 2. Therefore, under the ℓ_2 loss, the correct measure of the local communication complexity at p is its Rényi entropy.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/16/2021

Breaking The Dimension Dependence in Sparse Distribution Estimation under Communication Constraints

We consider the problem of estimating a d-dimensional s-sparse discrete ...
research
01/25/2019

Communication Complexity of Estimating Correlations

We characterize the communication complexity of the following distribute...
research
05/25/2021

On learning parametric distributions from quantized samples

We consider the problem of learning parametric distributions from their ...
research
02/07/2019

Learning Distributions from their Samples under Communication Constraints

We consider the problem of learning high-dimensional, nonparametric and ...
research
04/21/2021

Robust Testing and Estimation under Manipulation Attacks

We study robust testing and estimation of discrete distributions in the ...
research
08/31/2022

Fine-Grained Distribution-Dependent Learning Curves

Learning curves plot the expected error of a learning algorithm as a fun...
research
01/24/2020

Distributed Gaussian Mean Estimation under Communication Constraints: Optimal Rates and Communication-Efficient Algorithms

We study distributed estimation of a Gaussian mean under communication c...

Please sign up or login with your details

Forgot password? Click here to reset