A general method for regularizing tensor decomposition methods via pseudo-data

05/24/2019
by   Omer Gottesman, et al.
0

Tensor decomposition methods allow us to learn the parameters of latent variable models through decomposition of low-order moments of data. A significant limitation of these algorithms is that there exists no general method to regularize them, and in the past regularization has mostly been performed using bespoke modifications to the algorithms, tailored for the particular form of the desired regularizer. We present a general method of regularizing tensor decomposition methods which can be used for any likelihood model that is learnable using tensor decomposition methods and any differentiable regularization function by supplementing the training data with pseudo-data. The pseudo-data is optimized to balance two terms: being as close as possible to the true data and enforcing the desired regularization. On synthetic, semi-synthetic and real data, we demonstrate that our method can improve inference accuracy and regularize for a broad range of goals including transfer learning, sparsity, interpretability, and orthogonality of the learned parameters.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/17/2018

Hierarchical Methods of Moments

Spectral methods of moments provide a powerful tool for learning the par...
research
02/19/2023

Noise-Augmented ℓ_0 Regularization of Tensor Regression with Tucker Decomposition

Tensor data are multi-dimension arrays. Low-rank decomposition-based reg...
research
01/29/2019

Fast Randomized Matrix and Tensor Interpolative Decomposition Using CountSketch

We propose a new fast randomized algorithm for interpolative decompositi...
research
08/16/2019

Pseudo-task Regularization for ConvNet Transfer Learning

This paper is about regularizing deep convolutional networks (ConvNets) ...
research
12/27/2018

Sparse Nonnegative CANDECOMP/PARAFAC Decomposition in Block Coordinate Descent Framework: A Comparison Study

Nonnegative CANDECOMP/PARAFAC (NCP) decomposition is an important tool t...
research
03/10/2019

Algorithms for an Efficient Tensor Biclustering

Consider a data set collected by (individuals-features) pairs in differe...
research
10/18/2017

Weighted Tensor Decomposition for Learning Latent Variables with Partial Data

Tensor decomposition methods are popular tools for learning latent varia...

Please sign up or login with your details

Forgot password? Click here to reset