Wearing a MASK: Compressed Representations of Variable-Length Sequences Using Recurrent Neural Tangent Kernels

10/27/2020
by   Sina Alemohammad, et al.
0

High dimensionality poses many challenges to the use of data, from visualization and interpretation, to prediction and storage for historical preservation. Techniques abound to reduce the dimensionality of fixed-length sequences, yet these methods rarely generalize to variable-length sequences. To address this gap, we extend existing methods that rely on the use of kernels to variable-length sequences via use of the Recurrent Neural Tangent Kernel (RNTK). Since a deep neural network with ReLu activation is a Max-Affine Spline Operator (MASO), we dub our approach Max-Affine Spline Kernel (MASK). We demonstrate how MASK can be used to extend principal components analysis (PCA) and t-distributed stochastic neighbor embedding (t-SNE) and apply these new algorithms to separate synthetic time series data sampled from second-order differential equations.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

09/09/2011

Learning Sequence Neighbourhood Metrics

Recurrent neural networks (RNNs) in combination with a pooling operator ...
06/07/2019

Recurrent Kernel Networks

Substring kernels are classical tools for representing biological sequen...
06/18/2020

The Recurrent Neural Tangent Kernel

The study of deep networks (DNs) in the infinite-width limit, via the so...
05/23/2021

On the preservation of second integrals by Runge-Kutta methods

One can elucidate integrability properties of ordinary differential equa...
06/21/2021

Neural Controlled Differential Equations for Online Prediction Tasks

Neural controlled differential equations (Neural CDEs) are a continuous-...
10/20/2019

Rational Kernels: A survey

Many kinds of data are naturally amenable to being treated as sequences....
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.