Invertible Low-Divergence Coding

10/20/2020
by   Patrick Schulte, et al.
0

Several applications in communication, control, and learning require approximating target distributions to within small informational divergence (I-divergence). The additional requirement of invertibility usually leads to using encoders that are one-to-one mappings, also known as distribution matchers. However, even the best one-to-one encoders have I-divergences that grow logarithmically with the block length in general. To improve performance, an encoder is proposed that has an invertible one-to-many mapping and a low-rate resolution code. Two algorithms are developed to design the mapping by assigning strings in either a most-likely first or least-likely first order. Both algorithms give information rates approaching the entropy of the target distribution with exponentially decreasing I-divergence and with vanishing resolution rate in the block length.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/02/2021

Divergence Scaling for Distribution Matching

Distribution matchers for finite alphabets are shown to have information...
research
03/31/2021

α-Geodesical Skew Divergence

The asymmetric skew divergence smooths one of the distributions by mixin...
research
03/09/2018

Shell Mapping for Distribution matching

Distribution matching transforms independent and Bernoulli(1/2) distribu...
research
04/03/2019

Arithmetic Coding Based Multi-Composition Codes for Bit-Level Distribution Matching

A distribution matcher (DM) encodes a binary input data sequence into a ...
research
10/19/2020

On the Difficulty of Unbiased Alpha Divergence Minimization

Several approximate inference algorithms have been proposed to minimize ...
research
05/11/2020

On Parity-Preserving Variable-Length Constrained Coding

Previous work by the authors on parity-preserving fixed-length constrain...
research
05/09/2012

Multiple Source Adaptation and the Renyi Divergence

This paper presents a novel theoretical study of the general problem of ...

Please sign up or login with your details

Forgot password? Click here to reset