Compressing Multisets with Large Alphabets

07/15/2021
by   Daniel Severo, et al.
0

Current methods that optimally compress multisets are not suitable for high-dimensional symbols, as their compute time scales linearly with alphabet size. Compressing a multiset as an ordered sequence with off-the-shelf codecs is computationally more efficient, but has a sub-optimal compression rate, as bits are wasted encoding the order between symbols. We present a method that can recover those bits, assuming symbols are i.i.d., at the cost of an additional 𝒪(|ℳ|log M) in average time complexity, where |ℳ| and M are the total and unique number of symbols in the multiset. Our method is compatible with any prefix-free code. Experiments show that, when paired with efficient coders, our method can efficiently compress high-dimensional sources such as multisets of images and collections of JSON files.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/28/2018

On the number of symbols that forces a transversal

Akbari and Alipour conjectured that any Latin array of order n with at l...
research
04/29/2021

Causal (Progressive) Encoding over BinarySymmetric Channels with Noiseless Feedback

Traditional channel coding with feedback constructs and transmits a code...
research
01/13/2021

Improving Run Length Encoding by Preprocessing

The Run Length Encoding (RLE) compression method is a long standing simp...
research
01/15/2018

Efficient Trimmed Convolutional Arithmetic Encoding for Lossless Image Compression

Arithmetic encoding is an essential class of coding techniques which hav...
research
05/21/2021

Weighted Burrows-Wheeler Compression

A weight based dynamic compression method has recently been proposed, wh...
research
04/13/2022

On Extremal Rates of Secure Storage over Graphs

A secure storage code maps K source symbols, each of L_w bits, to N code...
research
08/25/2021

Encoding Scheme For Infinite Set of Symbols: The Percolation Process

It is shown here that the percolation process on binary trees that is eq...

Please sign up or login with your details

Forgot password? Click here to reset