The Foes of Neural Network's Data Efficiency Among Unnecessary Input Dimensions

07/13/2021
by   Vanessa D'Amario, et al.
0

Datasets often contain input dimensions that are unnecessary to predict the output label, e.g. background in object recognition, which lead to more trainable parameters. Deep Neural Networks (DNNs) are robust to increasing the number of parameters in the hidden layers, but it is unclear whether this holds true for the input layer. In this letter, we investigate the impact of unnecessary input dimensions on a central issue of DNNs: their data efficiency, ie. the amount of examples needed to achieve certain generalization performance. Our results show that unnecessary input dimensions that are task-unrelated substantially degrade data efficiency. This highlights the need for mechanisms that remove task-unrelated dimensions to enable data efficiency gains.

READ FULL TEXT
research
09/12/2022

Deep Neural Networks as Complex Networks

Deep Neural Networks are, from a physical perspective, graphs whose `lin...
research
07/07/2020

SpinalNet: Deep Neural Network with Gradual Input

Over the past few years, deep neural networks (DNNs) have garnered remar...
research
05/20/2022

The developmental trajectory of object recognition robustness: children are like small adults but unlike big deep neural networks

In laboratory object recognition tasks based on undistorted photographs,...
research
06/12/2014

Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition

The primate visual system achieves remarkable visual object recognition ...
research
10/21/2019

CPWC: Contextual Point Wise Convolution for Object Recognition

Convolutional layers are a major driving force behind the successes of d...
research
10/11/2020

Is It Time to Redefine the Classification Task for Deep Neural Networks?

Deep neural networks (DNNs) is demonstrated to be vulnerable to the adve...
research
03/09/2023

PDSketch: Integrated Planning Domain Programming and Learning

This paper studies a model learning and online planning approach towards...

Please sign up or login with your details

Forgot password? Click here to reset