Exponential Machines

by   Alexander Novikov, et al.
HSE University
Moscow Institute of Physics and Technology

Modeling interactions between features improves the performance of machine learning solutions in many domains (e.g. recommender systems or sentiment analysis). In this paper, we introduce Exponential Machines (ExM), a predictor that models all interactions of every order. The key idea is to represent an exponentially large tensor of parameters in a factorized format called Tensor Train (TT). The Tensor Train format regularizes the model and lets you control the number of underlying parameters. To train the model, we develop a stochastic Riemannian optimization procedure, which allows us to fit tensors with 2^160 entries. We show that the model achieves state-of-the-art performance on synthetic data with high-order interactions and that it works on par with high-order factorization machines on a recommender system dataset MovieLens 100K.


page 1

page 2

page 3

page 4


Tensor train construction from tensor actions, with application to compression of large high order derivative tensors

We present a method for converting tensors into tensor train format base...

Parallel algorithms for computing the tensor-train decomposition

The tensor-train (TT) decomposition expresses a tensor in a data-sparse ...

Tensor Decomposition via Variational Auto-Encoder

Tensor decomposition is an important technique for capturing the high-or...

Provable Tensor-Train Format Tensor Completion by Riemannian Optimization

The tensor train (TT) format enjoys appealing advantages in handling str...

Serialized Interacting Mixed Membership Stochastic Block Model

Last years have seen a regain of interest for the use of stochastic bloc...

Exact Partitioning of High-order Planted Models with a Tensor Nuclear Norm Constraint

We study the problem of efficient exact partitioning of the hypergraphs ...

Tensor-based Nonlinear Classifier for High-Order Data Analysis

In this paper we propose a tensor-based nonlinear model for high-order d...

Code Repositories


Tensor network machine learning. Based on the paper "Supervised Learning with Quantum Inspired Tensor Networks" http://arxiv.org/abs/1605.05775

view repo


Exponential Machines implementation

view repo

Please sign up or login with your details

Forgot password? Click here to reset