¶ILCRO: Making Importance Landscapes Flat Again

by   Vincent Moens, et al.

Convolutional neural networks have had a great success in numerous tasks, including image classification, object detection, sequence modelling, and many more. It is generally assumed that such neural networks are translation invariant, meaning that they can detect a given feature independent of its location in the input image. While this is true for simple cases, where networks are composed of a restricted number of layer classes and where images are fairly simple, complex images with common state-of-the-art networks do not usually enjoy this property as one might hope. This paper shows that most of the existing convolutional architectures define, at initialisation, a specific feature importance landscape that conditions their capacity to attend to different locations of the images later during training or even at test time. We demonstrate how this phenomenon occurs under specific conditions and how it can be adjusted under some assumptions. We derive the P-objective, or PILCRO for Pixel-wise Importance Landscape Curvature Regularised Objective, a simple regularisation technique that favours weight configurations that produce smooth, low-curvature importance landscapes that are conditioned on the data and not on the chosen architecture. Through extensive experiments, we further show that P-regularised versions of popular computer vision networks have a flat importance landscape, train faster, result in a better accuracy and are more robust to noise at test time, when compared to their original counterparts in common computer-vision classification settings.


page 1

page 2

page 3

page 4


Investigating Learning in Deep Neural Networks using Layer-Wise Weight Change

Understanding the per-layer learning dynamics of deep neural networks is...

SPIN: Simplifying Polar Invariance for Neural networks Application to vision-based irradiance forecasting

Translational invariance induced by pooling operations is an inherent pr...

CapsNet comparative performance evaluation for image classification

Image classification has become one of the main tasks in the field of co...

Hierarchical Deep Convolutional Neural Networks for Multi-category Diagnosis of Gastrointestinal Disorders on Histopathological Images

Deep convolutional neural networks (CNNs) have been successful for a wid...

Pixel-wise Ear Detection with Convolutional Encoder-Decoder Networks

Object detection and segmentation represents the basis for many tasks in...

Data Representations' Study of Latent Image Manifolds

Deep neural networks have been demonstrated to achieve phenomenal succes...

Please sign up or login with your details

Forgot password? Click here to reset