End-to-end Learning of a Convolutional Neural Network via Deep Tensor Decomposition

05/16/2018
by   Samet Oymak, et al.
0

In this paper we study the problem of learning the weights of a deep convolutional neural network. We consider a network where convolutions are carried out over non-overlapping patches with a single kernel in each layer. We develop an algorithm for simultaneously learning all the kernels from the training data. Our approach dubbed Deep Tensor Decomposition (DeepTD) is based on a rank-1 tensor decomposition. We theoretically investigate DeepTD under a realizable model for the training data where the inputs are chosen i.i.d. from a Gaussian distribution and the labels are generated according to planted convolutional kernels. We show that DeepTD is data-efficient and provably works as soon as the sample size exceeds the total number of convolutional weights in the network. We carry out a variety of numerical experiments to investigate the effectiveness of DeepTD and verify our theoretical findings.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/12/2020

Stable Low-rank Tensor Decomposition for Compression of Convolutional Neural Network

Most state of the art deep neural networks are overparameterized and exh...
research
02/20/2018

On the Connection Between Learning Two-Layers Neural Networks and Tensor Decomposition

We establish connections between the problem of learning a two-layers ne...
research
06/14/2019

Efficient N-Dimensional Convolutions via Higher-Order Factorization

With the unprecedented success of deep convolutional neural networks cam...
research
12/08/2019

Lossless Compression for 3DCNNs Based on Tensor Train Decomposition

Three dimensional convolutional neural networks (3DCNNs) have been appli...
research
06/29/2020

Hybrid Tensor Decomposition in Neural Network Compression

Deep neural networks (DNNs) have enabled impressive breakthroughs in var...
research
07/18/2017

Solving ℓ^p-norm regularization with tensor kernels

In this paper, we discuss how a suitable family of tensor kernels can be...
research
05/20/2018

Improved Learning of One-hidden-layer Convolutional Neural Networks with Overlaps

We propose a new algorithm to learn a one-hidden-layer convolutional neu...

Please sign up or login with your details

Forgot password? Click here to reset