A Machine Learning Perspective on Predictive Coding with PAQ

08/16/2011
by   Byron Knoll, et al.
0

PAQ8 is an open source lossless data compression algorithm that currently achieves the best compression rates on many benchmarks. This report presents a detailed description of PAQ8 from a statistical machine learning perspective. It shows that it is possible to understand some of the modules of PAQ8 and use this understanding to improve the method. However, intuitive statistical explanations of the behavior of other modules remain elusive. We hope the description in this report will be a starting point for discussions that will increase our understanding, lead to improvements to PAQ8, and facilitate a transfer of knowledge from PAQ8 to other machine learning methods, such a recurrent neural networks and stochastic memoizers. Finally, the report presents a broad range of new applications of PAQ to machine learning tasks including language modeling and adaptive text prediction, adaptive game playing, classification, and compression using features from the field of deep learning.

READ FULL TEXT

page 20

page 21

page 23

page 24

page 25

page 26

page 27

research
02/14/2022

An Introduction to Neural Data Compression

Neural compression is the application of neural networks and other machi...
research
01/05/2022

Understanding Entropy Coding With Asymmetric Numeral Systems (ANS): a Statistician's Perspective

Entropy coding is the backbone data compression. Novel machine-learning ...
research
02/06/2019

Compression of Recurrent Neural Networks for Efficient Language Modeling

Recurrent neural networks have proved to be an effective method for stat...
research
06/18/2017

Learning Hierarchical Information Flow with Recurrent Neural Modules

We propose ThalNet, a deep learning model inspired by neocortical commun...
research
10/31/2022

A Machine Learning Tutorial for Operational Meteorology, Part II: Neural Networks and Deep Learning

Over the past decade the use of machine learning in meteorology has grow...
research
10/08/2018

Neural Networks Models for Analyzing Magic: the Gathering Cards

Historically, games of all kinds have often been the subject of study in...
research
12/21/2020

SARS-CoV-2 Coronavirus Data Compression Benchmark

This paper introduces a lossless data compression competition that bench...

Please sign up or login with your details

Forgot password? Click here to reset