Log In Sign Up

Adaptive Tensor Learning with Tensor Networks

by   Meraj Hashemizadeh, et al.

Tensor decomposition techniques have shown great successes in machine learning and data science by extending classical algorithms based on matrix factorization to multi-modal and multi-way data. However, there exist many tensor decomposition models (CP, Tucker, Tensor Train, etc.) and the rank of such a decomposition is typically a collection of integers rather than a unique number, making model and hyper-parameter selection a tedious and costly task. At the same time, tensor network methods are powerful tools developed in the physics community which have recently shown their potential for machine learning applications and offer a unifying view of the various tensor decomposition models. In this paper, we leverage the tensor network formalism to develop a generic and efficient adaptive algorithm for tensor learning. Our method is based on a simple greedy approach optimizing a differentiable loss function starting from a rank one tensor and successively identifying the most promising tensor network edges for small rank increments. Our algorithm can adaptively identify tensor network structures with small number of parameters that effectively optimize the objective function from data. The framework we introduce is very broad and encompasses many common tensor optimization problems. Experiments on tensor decomposition and tensor completion tasks with both synthetic and real world data demonstrate the effectiveness of the proposed algorithm.


page 9

page 13


Alternating minimization algorithms for graph regularized tensor completion

We consider a low-rank tensor completion (LRTC) problem which aims to re...

Multi-Slice Clustering for 3-order Tensor Data

Several methods of triclustering of three dimensional data require the s...

Tensor networks in machine learning

A tensor network is a type of decomposition used to express and approxim...

A general method for regularizing tensor decomposition methods via pseudo-data

Tensor decomposition methods allow us to learn the parameters of latent ...

A Coupled CP Decomposition for Principal Components Analysis of Symmetric Networks

In a number of application domains, one observes a sequence of network d...

Rank-R FNN: A Tensor-Based Learning Model for High-Order Data Classification

An increasing number of emerging applications in data science and engine...