DeepAI AI Chat
Log In Sign Up

Lossless Compression for 3DCNNs Based on Tensor Train Decomposition

12/08/2019
by   Dingheng Wang, et al.
Tsinghua University
27

Three dimensional convolutional neural networks (3DCNNs) have been applied in many tasks of video or 3D point cloud recognition. However, due to the higher dimension of convolutional kernels, the space complexity of 3DCNNs is generally larger than that of traditional two dimensional convolutional neural networks (2DCNNs). To miniaturize 3DCNNs for the deployment in confining environments such as embedded devices, neural network compression is a promising approach. In this work, we adopt the tensor train (TT) decomposition, the most compact and simplest in situ training compression method, to shrink the 3DCNN models. We give the tensorizing for 3D convolutional kernels in TT format and investigate how to select appropriate ranks for the tensor in TT format. In the light of multiple contrast experiments based on VIVA challenge and UCF11 datasets, we conclude that the TT decomposition can compress redundant 3DCNNs in a ratio up to 121× with little accuracy improvement. Besides, we achieve a state-of-the-art result of TT-3DCNN on VIVA challenge dataset (81.83%).

READ FULL TEXT

page 7

page 8

page 10

page 12

06/29/2020

Hybrid Tensor Decomposition in Neural Network Compression

Deep neural networks (DNNs) have enabled impressive breakthroughs in var...
07/26/2021

Towards Efficient Tensor Decomposition-Based DNN Model Compression with Optimization Framework

Advanced tensor decomposition, such as Tensor train (TT) and Tensor ring...
04/11/2023

A priori compression of convolutional neural networks for wave simulators

Convolutional neural networks are now seeing widespread use in a variety...
05/16/2018

End-to-end Learning of a Convolutional Neural Network via Deep Tensor Decomposition

In this paper we study the problem of learning the weights of a deep con...
05/24/2019

Bayesian Tensorized Neural Networks with Automatic Rank Selection

Tensor decomposition is an effective approach to compress over-parameter...
11/22/2022

An Incremental Tensor Train Decomposition Algorithm

We present a new algorithm for incrementally updating the tensor-train d...