Reduced-Order Modeling of Deep Neural Networks

10/15/2019
by   Talgat Daulbaev, et al.
0

We introduce a new method for speeding up the inference of deep neural networks. It is somewhat inspired by the reduced-order modeling techniques for dynamical systems. The cornerstone of the proposed method is the maximum volume algorithm. We demonstrate efficiency on VGG and ResNet architectures pre-trained on different datasets. We show that in many practical cases it is possible to replace convolutional layers with much smaller fully-connected layers with a relatively small drop in accuracy.

READ FULL TEXT
research
12/22/2014

Deep Fried Convnets

The fully connected layers of a deep convolutional neural network typica...
research
10/06/2021

Characterizing Learning Dynamics of Deep Neural Networks via Complex Networks

In this paper, we interpret Deep Neural Networks with Complex Network Th...
research
05/14/2018

Unifying and Merging Well-trained Deep Neural Networks for Inference Stage

We propose a novel method to merge convolutional neural-nets for the inf...
research
11/03/2021

On the Application of Data-Driven Deep Neural Networks in Linear and Nonlinear Structural Dynamics

The use of deep neural network (DNN) models as surrogates for linear and...
research
02/25/2018

Wide Compression: Tensor Ring Nets

Deep neural networks have demonstrated state-of-the-art performance in a...
research
03/25/2022

Deformable Butterfly: A Highly Structured and Sparse Linear Transform

We introduce a new kind of linear transform named Deformable Butterfly (...
research
07/13/2019

Towards Characterizing and Limiting Information Exposure in DNN Layers

Pre-trained Deep Neural Network (DNN) models are increasingly used in sm...

Please sign up or login with your details

Forgot password? Click here to reset