Tensor Regression Networks with various Low-Rank Tensor Approximations

12/27/2017
by   Xingwei Cao, et al.
0

Tensor regression networks achieve high rate of compression of model parameters in multilayer perceptrons (MLP) while having slight impact on performances. Tensor regression layer imposes low-rank constraints on the tensor regression layer which replaces the flattening operation of traditional MLP. We investigate tensor regression networks using various low-rank tensor approximations, aiming to leverage the multi-modal structure of high dimensional data by enforcing efficient low-rank constraints. We provide a theoretical analysis giving insights on the choice of the rank parameters. We evaluated performance of proposed model with state-of-the-art deep convolutional models. For CIFAR-10 dataset, we achieved the compression rate of 0.018 with the sacrifice of accuracy less than 1

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/08/2022

Almost-lossless compression of a low-rank random tensor

In this work, we establish an asymptotic limit of almost-lossless compre...
research
03/08/2021

Low-Rank Tensor Regression for X-Ray Tomography

Tomographic imaging is useful for revealing the internal structure of a ...
research
11/18/2018

The core consistency of a compressed tensor

Tensor decomposition on big data has attracted significant attention rec...
research
02/27/2019

Stochastically Rank-Regularized Tensor Regression Networks

Over-parametrization of deep neural networks has recently been shown to ...
research
05/16/2023

Errors-in-variables Fréchet Regression with Low-rank Covariate Approximation

Fréchet regression has emerged as a promising approach for regression an...
research
10/31/2017

Tensor Regression Meets Gaussian Processes

Low-rank tensor regression, a new model class that learns high-order cor...
research
03/31/2021

High-Dimensional Uncertainty Quantification via Rank- and Sample-Adaptive Tensor Regression

Fabrication process variations can significantly influence the performan...

Please sign up or login with your details

Forgot password? Click here to reset