Tensor-Train Networks for Learning Predictive Modeling of Multidimensional Data

by   M. Nazareth da Costa, et al.

Deep neural networks have attracted the attention of the machine learning community because of their appealing data-driven framework and of their performance in several pattern recognition tasks. On the other hand, there are many open theoretical problems regarding the internal operation of the network, the necessity of certain layers, hyperparameter selection etc. A promising strategy is based on tensor networks, which have been very successful in physical and chemical applications. In general, higher-order tensors are decomposed into sparsely interconnected lower-order tensors. This is a numerically reliable way to avoid the curse of dimensionality and to provide highly compressed representation of a data tensor, besides the good numerical properties that allow to control the desired accuracy of approximation. In order to compare tensor and neural networks, we first consider the identification of the classical Multilayer Perceptron using Tensor-Train. A comparative analysis is also carried out in the context of prediction of the Mackey-Glass noisy chaotic time series and NASDAQ index. We have shown that the weights of a multidimensional regression model can be learned by means of tensor networks with the aim of performing a powerful compact representation retaining the accuracy of neural networks. Furthermore, an algorithm based on alternating least squares has been proposed for approximating the weights in TT-format with a reduction of computational calculus. By means of a direct expression, we have approximated the core estimation as the conventional solution for a general regression model, which allows to extend the applicability of tensor structures to different algorithms.


page 1

page 2

page 3

page 4


An efficient tensor regression for high-dimensional data

Most currently used tensor regression models for high-dimensional data a...

TLib: A Flexible C++ Tensor Framework for Numerical Tensor Calculus

Numerical tensor calculus comprise basic tensor operations such as the e...

Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives

Part 2 of this monograph builds on the introduction to tensor networks a...

Black box approximation in the tensor train format initialized by ANOVA decomposition

Surrogate models can reduce computational costs for multivariable functi...

On some orthogonalization schemes in Tensor Train format

In the framework of tensor spaces, we consider orthogonalization kernels...

Edge-based Tensor prediction via graph neural networks

Message-passing neural networks (MPNN) have shown extremely high efficie...

Higher-Order Partial Least Squares (HOPLS): A Generalized Multi-Linear Regression Method

A new generalized multilinear regression model, termed the Higher-Order ...

Please sign up or login with your details

Forgot password? Click here to reset