Conflicting Bundles: Adapting Architectures Towards the Improved Training of Deep Neural Networks

11/05/2020 ∙ by David Peer, et al. ∙ 0

Designing neural network architectures is a challenging task and knowing which specific layers of a model must be adapted to improve the performance is almost a mystery. In this paper, we introduce a novel theory and metric to identify layers that decrease the test accuracy of the trained models, this identification is done as early as at the beginning of training. In the worst-case, such a layer could lead to a network that can not be trained at all. More precisely, we identified those layers that worsen the performance because they produce conflicting training bundles as we show in our novel theoretical analysis, complemented by our extensive empirical studies. Based on these findings, a novel algorithm is introduced to remove performance decreasing layers automatically. Architectures found by this algorithm achieve a competitive accuracy when compared against the state-of-the-art architectures. While keeping such high accuracy, our approach drastically reduces memory consumption and inference time for different computer vision tasks.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 6

Code Repositories

conflicting-bundles

Official implementation of the paper "Conflicting Bundles: Adapting Architectures Towards the Improved Training of Deep Neural Networks".


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.