On Distributed Learning with Constant Communication Bits

by   Xiangxiang Xu, et al.

In this paper, we study a distributed learning problem constrained by constant communication bits. Specifically, we consider the distributed hypothesis testing (DHT) problem where two distributed nodes are constrained to transmit a constant number of bits to a central decoder. In such cases, we show that in order to achieve the optimal error exponents, it suffices to consider the empirical distributions of observed data sequences and encode them to the transmission bits. With such a coding strategy, we develop a geometric approach in the distribution spaces and characterize the optimal schemes. In particular, we show the optimal achievable error exponents and coding schemes for the following cases: (i) both nodes can transmit log_23 bits; (ii) one of the nodes can transmit 1 bit, and the other node is not constrained; (iii) the joint distribution of the nodes are conditionally independent given one hypothesis. Furthermore, we provide several numerical examples for illustrating the theoretical results. Our results provide theoretical guidance for designing practical distributed learning rules, and the developed approach also reveals new potentials for establishing error exponents for DHT with more general communication constraints.



page 1

page 2

page 3

page 4


Learning Distributions from their Samples under Communication Constraints

We consider the problem of learning high-dimensional, nonparametric and ...

Distributed Hypothesis Testing Over Discrete Memoryless Channels

A distributed binary hypothesis testing problem is studied in which mult...

On Distributed Quantization for Classification

We consider the problem of distributed feature quantization, where the g...

An Information-theoretic Method for Collaborative Distributed Learning with Limited Communication

In this paper, we study the information transmission problem under the d...

Communication Complexity of Distributed High Dimensional Correlation Testing

Two parties observe independent copies of a d-dimensional vector and a s...

Coding for Crowdsourced Classification with XOR Queries

This paper models the crowdsourced labeling/classification problem as a ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.