DeepAI AI Chat
Log In Sign Up

From Imitation to Prediction, Data Compression vs Recurrent Neural Networks for Natural Language Processing

by   Juan Andrés Laura, et al.
University of Buenos Aires

In recent studies [1][13][12] Recurrent Neural Networks were used for generative processes and their surprising performance can be explained by their ability to create good predictions. In addition, data compression is also based on predictions. What the problem comes down to is whether a data compressor could be used to perform as well as recurrent neural networks in natural language processing tasks. If this is possible,then the problem comes down to determining if a compression algorithm is even more intelligent than a neural network in specific tasks related to human language. In our journey we discovered what we think is the fundamental difference between a Data Compression Algorithm and a Recurrent Neural Network.


page 1

page 2

page 3

page 4


Using Recurrent Neural Network for Learning Expressive Ontologies

Recently, Neural Networks have been proven extremely effective in many n...

DeepZip: Lossless Data Compression using Recurrent Neural Networks

Sequential data is being generated at an unprecedented pace in various f...

Verifying Recurrent Neural Networks using Invariant Inference

Deep neural networks are revolutionizing the way complex systems are dev...

Recurrent Neural Networks to automate Quality assessment of Software Requirements

Many problems related to the quality of requirements arise during elicit...

Character-level Recurrent Neural Networks in Practice: Comparing Training and Sampling Schemes

Recurrent neural networks are nowadays successfully used in an abundance...

Syntactically Informed Text Compression with Recurrent Neural Networks

We present a self-contained system for constructing natural language mod...