One time is not enough: iterative tensor decomposition for neural network compression

03/24/2019
by   Julia Gusak, et al.
24

The low-rank tensor approximation is very promising for the compression of deep neural networks. We propose a new simple and efficient iterative approach, which alternates low-rank factorization with a smart rank selection and fine-tuning. We demonstrate the efficiency of our method comparing to non-iterative ones. Our approach improves the compression rate while maintaining the accuracy for a variety of tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/08/2022

Almost-lossless compression of a low-rank random tensor

In this work, we establish an asymptotic limit of almost-lossless compre...
research
08/12/2020

Stable Low-rank Tensor Decomposition for Compression of Convolutional Neural Network

Most state of the art deep neural networks are overparameterized and exh...
research
03/23/2018

Iterative Low-Rank Approximation for CNN Compression

Deep convolutional neural networks contain tens of millions of parameter...
research
11/02/2021

Low-Rank+Sparse Tensor Compression for Neural Networks

Low-rank tensor compression has been proposed as a promising approach to...
research
07/30/2022

Distilled Low Rank Neural Radiance Field with Quantization for Light Field Compression

In this paper, we propose a novel light field compression method based o...
research
09/04/2017

Domain-adaptive deep network compression

Deep Neural Networks trained on large datasets can be easily transferred...
research
03/12/2019

Cascaded Projection: End-to-End Network Compression and Acceleration

We propose a data-driven approach for deep convolutional neural network ...

Please sign up or login with your details

Forgot password? Click here to reset