Lossless Compression for 3DCNNs Based on Tensor Train Decomposition

12/08/2019
by   Dingheng Wang, et al.
27

Three dimensional convolutional neural networks (3DCNNs) have been applied in many tasks of video or 3D point cloud recognition. However, due to the higher dimension of convolutional kernels, the space complexity of 3DCNNs is generally larger than that of traditional two dimensional convolutional neural networks (2DCNNs). To miniaturize 3DCNNs for the deployment in confining environments such as embedded devices, neural network compression is a promising approach. In this work, we adopt the tensor train (TT) decomposition, the most compact and simplest in situ training compression method, to shrink the 3DCNN models. We give the tensorizing for 3D convolutional kernels in TT format and investigate how to select appropriate ranks for the tensor in TT format. In the light of multiple contrast experiments based on VIVA challenge and UCF11 datasets, we conclude that the TT decomposition can compress redundant 3DCNNs in a ratio up to 121× with little accuracy improvement. Besides, we achieve a state-of-the-art result of TT-3DCNN on VIVA challenge dataset (81.83%).

READ FULL TEXT

page 7

page 8

page 10

page 12

research
06/29/2020

Hybrid Tensor Decomposition in Neural Network Compression

Deep neural networks (DNNs) have enabled impressive breakthroughs in var...
research
07/26/2021

Towards Efficient Tensor Decomposition-Based DNN Model Compression with Optimization Framework

Advanced tensor decomposition, such as Tensor train (TT) and Tensor ring...
research
08/10/2021

Tensor Yard: One-Shot Algorithm of Hardware-Friendly Tensor-Train Decomposition for Convolutional Neural Networks

Nowadays Deep Learning became widely used in many economic, technical an...
research
04/11/2023

A priori compression of convolutional neural networks for wave simulators

Convolutional neural networks are now seeing widespread use in a variety...
research
05/16/2018

End-to-end Learning of a Convolutional Neural Network via Deep Tensor Decomposition

In this paper we study the problem of learning the weights of a deep con...
research
05/24/2019

Bayesian Tensorized Neural Networks with Automatic Rank Selection

Tensor decomposition is an effective approach to compress over-parameter...
research
11/22/2022

An Incremental Tensor Train Decomposition Algorithm

We present a new algorithm for incrementally updating the tensor-train d...

Please sign up or login with your details

Forgot password? Click here to reset