DeepAI AI Chat
Log In Sign Up

Lempel-Ziv Networks

by   Rebecca Saul, et al.

Sequence processing has long been a central area of machine learning research. Recurrent neural nets have been successful in processing sequences for a number of tasks; however, they are known to be both ineffective and computationally expensive when applied to very long sequences. Compression-based methods have demonstrated more robustness when processing such sequences – in particular, an approach pairing the Lempel-Ziv Jaccard Distance (LZJD) with the k-Nearest Neighbor algorithm has shown promise on long sequence problems (up to T=200,000,000 steps) involving malware classification. Unfortunately, use of LZJD is limited to discrete domains. To extend the benefits of LZJD to a continuous domain, we investigate the effectiveness of a deep-learning analog of the algorithm, the Lempel-Ziv Network. While we achieve successful proof of concept, we are unable to improve meaningfully on the performance of a standard LSTM across a variety of datasets and sequence processing tasks. In addition to presenting this negative result, our work highlights the problem of sub-par baseline tuning in newer research areas.


page 1

page 2

page 3

page 4


Malware Detection by Eating a Whole EXE

In this work we introduce malware detection from raw byte sequences as a...

ACO Implementation for Sequence Alignment with Genetic Algorithms

In this paper, we implement Ant Colony Optimization (ACO) for sequence a...

Malware Classification with Word Embedding Features

Malware classification is an important and challenging problem in inform...

Efficient LSTM Training with Eligibility Traces

Training recurrent neural networks is predominantly achieved via backpro...

Actively Learning what makes a Discrete Sequence Valid

Deep learning techniques have been hugely successful for traditional sup...

Malware Classification with GMM-HMM Models

Discrete hidden Markov models (HMM) are often applied to malware detecti...

Latent Discretization for Continuous-time Sequence Compression

Neural compression offers a domain-agnostic approach to creating codecs ...