Unifying and Merging Well-trained Deep Neural Networks for Inference Stage

05/14/2018
by   Yi-Min Chou, et al.
0

We propose a novel method to merge convolutional neural-nets for the inference stage. Given two well-trained networks that may have different architectures that handle different tasks, our method aligns the layers of the original networks and merges them into a unified model by sharing the representative codes of weights. The shared weights are further re-trained to fine-tune the performance of the merged model. The proposed method effectively produces a compact model that may run original tasks simultaneously on resource-limited devices. As it preserves the general architectures and leverages the co-used weights of well-trained networks, a substantial training overhead can be reduced to shorten the system development time. Experimental results demonstrate a satisfactory performance and validate the effectiveness of the method.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/07/2021

GAN Cocktail: mixing GANs without dataset access

Today's generative models are capable of synthesizing high-fidelity imag...
research
10/15/2019

Reduced-Order Modeling of Deep Neural Networks

We introduce a new method for speeding up the inference of deep neural n...
research
04/15/2020

A Hybrid Method for Training Convolutional Neural Networks

Artificial Intelligence algorithms have been steadily increasing in popu...
research
10/14/2019

Variation-aware Binarized Memristive Networks

The quantization of weights to binary states in Deep Neural Networks (DN...
research
02/26/2019

Recurrent Convolution for Compact and Cost-Adjustable Neural Networks: An Empirical Study

Recurrent convolution (RC) shares the same convolutional kernels and unr...
research
01/21/2019

On Compression of Unsupervised Neural Nets by Pruning Weak Connections

Unsupervised neural nets such as Restricted Boltzmann Machines(RBMs) and...
research
01/17/2020

Adapting Grad-CAM for Embedding Networks

The gradient-weighted class activation mapping (Grad-CAM) method can fai...

Please sign up or login with your details

Forgot password? Click here to reset