Recurrent Kernel Networks

06/07/2019
by   Dexiong Chen, et al.
9

Substring kernels are classical tools for representing biological sequences or text. However, when large amounts of annotated data is available, models that allow end-to-end training such as neural networks are often prefered. Links between recurrent neural networks (RNNs) and substring kernels have recently been drawn, by formally showing that RNNs with specific activation functions were points in a reproducing kernel Hilbert space (RKHS). In this paper, we revisit this link by generalizing convolutional kernel networks---originally related to a relaxation of the mismatch kernel---to model gaps in sequences. It results in a new type of recurrent neural network which can be trained end-to-end with backpropagation, or without supervision by using kernel approximation techniques. We experimentally show that our approach is well suited to biological sequences, where it outperforms existing methods for protein classification tasks.

READ FULL TEXT

page 18

page 19

research
07/06/2018

Sliced Recurrent Neural Networks

Recurrent neural networks have achieved great success in many NLP tasks....
research
05/20/2016

End-to-End Kernel Learning with Supervised Convolutional Kernel Networks

In this paper, we introduce a new image representation based on a multil...
research
05/25/2017

Deriving Neural Architectures from Sequence and Graph Kernels

The design of neural architectures for structured objects is typically g...
research
10/27/2020

Wearing a MASK: Compressed Representations of Variable-Length Sequences Using Recurrent Neural Tangent Kernels

High dimensionality poses many challenges to the use of data, from visua...
research
01/21/2019

Training Neural Networks as Learning Data-adaptive Kernels: Provable Representation and Approximation Benefits

Consider the problem: given data pair (x, y) drawn from a population wit...
research
06/22/2020

Understanding Recurrent Neural Networks Using Nonequilibrium Response Theory

Recurrent neural networks (RNNs) are brain-inspired models widely used i...
research
04/06/2023

Biological Sequence Kernels with Guaranteed Flexibility

Applying machine learning to biological sequences - DNA, RNA and protein...

Please sign up or login with your details

Forgot password? Click here to reset