On Distributed Learning with Constant Communication Bits

09/14/2021
by   Xiangxiang Xu, et al.
0

In this paper, we study a distributed learning problem constrained by constant communication bits. Specifically, we consider the distributed hypothesis testing (DHT) problem where two distributed nodes are constrained to transmit a constant number of bits to a central decoder. In such cases, we show that in order to achieve the optimal error exponents, it suffices to consider the empirical distributions of observed data sequences and encode them to the transmission bits. With such a coding strategy, we develop a geometric approach in the distribution spaces and characterize the optimal schemes. In particular, we show the optimal achievable error exponents and coding schemes for the following cases: (i) both nodes can transmit log_23 bits; (ii) one of the nodes can transmit 1 bit, and the other node is not constrained; (iii) the joint distribution of the nodes are conditionally independent given one hypothesis. Furthermore, we provide several numerical examples for illustrating the theoretical results. Our results provide theoretical guidance for designing practical distributed learning rules, and the developed approach also reveals new potentials for establishing error exponents for DHT with more general communication constraints.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/24/2022

A Fundamental Limit of Distributed Hypothesis Testing Under Memoryless Quantization

We study a distributed hypothesis testing setup where peripheral nodes s...
research
02/07/2019

Learning Distributions from their Samples under Communication Constraints

We consider the problem of learning high-dimensional, nonparametric and ...
research
11/01/2019

On Distributed Quantization for Classification

We consider the problem of distributed feature quantization, where the g...
research
05/13/2022

An Information-theoretic Method for Collaborative Distributed Learning with Limited Communication

In this paper, we study the information transmission problem under the d...
research
05/21/2020

Communication Complexity of Distributed High Dimensional Correlation Testing

Two parties observe independent copies of a d-dimensional vector and a s...
research
04/05/2019

Optimal Communication Rates for Zero-Error Distributed Simulation under Blackboard Communication Protocols

We study the distributed simulation problem where n users aim to generat...

Please sign up or login with your details

Forgot password? Click here to reset