Residual Matrix Product State for Machine Learning

12/22/2020
by   Ye-Ming Meng, et al.
0

Tensor network (TN), which originates from quantum physics, shows broad prospects in classical and quantum machine learning (ML). However, there still exists a considerable gap of accuracy between TN and the sophisticated neural network (NN) models for classical ML. It is still elusive how far TN ML can be improved by, e.g., borrowing the techniques from NN. In this work, we propose the residual matrix product state (ResMPS) by combining the ideas of matrix product state (MPS) and residual NN. ResMPS can be treated as a network where its layers map the "hidden" features to the outputs (e.g., classifications), and the variational parameters of the layers are the functions of the features of samples (e.g., pixels of images). This is essentially different from NN, where the layers map feed-forwardly the features to the output. ResMPS can naturally incorporate with the non-linear activations and dropout layers, and outperforms the state-of-the-art TN models on the efficiency, stability, and expression power. Besides, ResMPS is interpretable from the perspective of polynomial expansion, where the factorization and exponential machines naturally emerge. Our work contributes to connecting and hybridizing neural and tensor networks, which is crucial to understand the working mechanisms further and improve both models' performances.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/10/2023

Compressing neural network by tensor network with exponentially fewer variational parameters

Neural network (NN) designed for challenging machine learning tasks is i...
research
06/24/2021

Tensor networks for unsupervised machine learning

Modeling the joint distribution of high-dimensional data is a central ta...
research
04/06/2018

Quantum Machine Learning Matrix Product States

Matrix product states minimize bipartite correlations to compress the cl...
research
01/02/2020

Visual Machine Learning: Insight through Eigenvectors, Chladni patterns and community detection in 2D particulate structures

Machine learning (ML) is quickly emerging as a powerful tool with divers...
research
03/18/2021

Neural tensor contractions and the expressive power of deep neural quantum states

We establish a direct connection between general tensor networks and dee...
research
11/19/2022

Class-Specific Attention (CSA) for Time-Series Classification

Most neural network-based classifiers extract features using several hid...
research
09/22/2020

Tensor Programs III: Neural Matrix Laws

In a neural network (NN), weight matrices linearly transform inputs into...

Please sign up or login with your details

Forgot password? Click here to reset