Investigating Learning in Deep Neural Networks using Layer-Wise Weight Change

11/13/2020 ∙ by Ayush Manish Agrawal, et al. ∙ 23

Understanding the per-layer learning dynamics of deep neural networks is of significant interest as it may provide insights into how neural networks learn and the potential for better training regimens. We investigate learning in Deep Convolutional Neural Networks (CNNs) by measuring the relative weight change of layers while training. Several interesting trends emerge in a variety of CNN architectures across various computer vision classification tasks, including the overall increase in relative weight change of later layers as compared to earlier ones.



There are no comments yet.


page 1

page 2

page 3

page 4

Code Repositories


The repository contains the codebase for Relative Weight Change in layers of Deep Neural Networks.

view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.