Generalization and Overfitting in Matrix Product State Machine Learning Architectures

08/08/2022
by   Artem Strashko, et al.
0

While overfitting and, more generally, double descent are ubiquitous in machine learning, increasing the number of parameters of the most widely used tensor network, the matrix product state (MPS), has generally lead to monotonic improvement of test performance in previous studies. To better understand the generalization properties of architectures parameterized by MPS, we construct artificial data which can be exactly modeled by an MPS and train the models with different number of parameters. We observe model overfitting for one-dimensional data, but also find that for more complex data overfitting is less significant, while with MNIST image data we do not find any signatures of overfitting. We speculate that generalization properties of MPS depend on the properties of data: with one-dimensional data (for which the MPS ansatz is the most suitable) MPS is prone to overfitting, while with more complex data which cannot be fit by MPS exactly, overfitting may be much less significant.

READ FULL TEXT
research
08/06/2020

Benign Overfitting and Noisy Features

Modern machine learning often operates in the regime where the number of...
research
06/01/2022

Realistic Deep Learning May Not Fit Benignly

Studies on benign overfitting provide insights for the success of overpa...
research
09/07/2022

Machine Learning Students Overfit to Overfitting

Overfitting and generalization is an important concept in Machine Learni...
research
08/16/2023

Quantifying Overfitting: Introducing the Overfitting Index

In the rapidly evolving domain of machine learning, ensuring model gener...
research
06/06/2021

Towards an Understanding of Benign Overfitting in Neural Networks

Modern machine learning models often employ a huge number of parameters ...
research
05/23/2022

Overfitting in quantum machine learning and entangling dropout

The ultimate goal in machine learning is to construct a model function t...
research
08/08/2019

Optimal multiclass overfitting by sequence reconstruction from Hamming queries

A primary concern of excessive reuse of test datasets in machine learnin...

Please sign up or login with your details

Forgot password? Click here to reset