DeepAI AI Chat
Log In Sign Up

From Imitation to Prediction, Data Compression vs Recurrent Neural Networks for Natural Language Processing

05/01/2017
by   Juan Andrés Laura, et al.
University of Buenos Aires
0

In recent studies [1][13][12] Recurrent Neural Networks were used for generative processes and their surprising performance can be explained by their ability to create good predictions. In addition, data compression is also based on predictions. What the problem comes down to is whether a data compressor could be used to perform as well as recurrent neural networks in natural language processing tasks. If this is possible,then the problem comes down to determining if a compression algorithm is even more intelligent than a neural network in specific tasks related to human language. In our journey we discovered what we think is the fundamental difference between a Data Compression Algorithm and a Recurrent Neural Network.

READ FULL TEXT

page 1

page 2

page 3

page 4

07/14/2016

Using Recurrent Neural Network for Learning Expressive Ontologies

Recently, Neural Networks have been proven extremely effective in many n...
11/20/2018

DeepZip: Lossless Data Compression using Recurrent Neural Networks

Sequential data is being generated at an unprecedented pace in various f...
04/06/2020

Verifying Recurrent Neural Networks using Invariant Inference

Deep neural networks are revolutionizing the way complex systems are dev...
05/11/2021

Recurrent Neural Networks to automate Quality assessment of Software Requirements

Many problems related to the quality of requirements arise during elicit...
01/02/2018

Character-level Recurrent Neural Networks in Practice: Comparing Training and Sampling Schemes

Recurrent neural networks are nowadays successfully used in an abundance...
08/08/2016

Syntactically Informed Text Compression with Recurrent Neural Networks

We present a self-contained system for constructing natural language mod...