DeepAI AI Chat
Log In Sign Up

A Unifying Framework of Bilinear LSTMs

by   Mohit Rajpal, et al.

This paper presents a novel unifying framework of bilinear LSTMs that can represent and utilize the nonlinear interaction of the input features present in sequence datasets for achieving superior performance over a linear LSTM and yet not incur more parameters to be learned. To realize this, our unifying framework allows the expressivity of the linear vs. bilinear terms to be balanced by correspondingly trading off between the hidden state vector size vs. approximation quality of the weight matrix in the bilinear term so as to optimize the performance of our bilinear LSTM, while not incurring more parameters to be learned. We empirically evaluate the performance of our bilinear LSTM in several language-based sequence learning tasks to demonstrate its general applicability.


On Bilinear Time Domain Identification

The Loewner framework (LF) in combination with Volterra series (VS) offe...

A unifying framework for tangential interpolation of structured bilinear control systems

In this paper, we consider the structure-preserving model order reductio...

Bilinear Recovery using Adaptive Vector-AMP

We consider the problem of jointly recovering the vector b and the matri...

A framework for fitting quadratic-bilinear systems with applications to models of electrical circuits

In this contribution, we propose a data-driven procedure to fit quadrati...

Embedding Entities and Relations for Learning and Inference in Knowledge Bases

We consider learning representations of entities and relations in KBs us...

Bilinear Attention Networks

Attention networks in multimodal learning provide an efficient way to ut...

A technical note on bilinear layers for interpretability

The ability of neural networks to represent more features than neurons m...