DeepAI AI Chat
Log In Sign Up

The Intrinsic Dimension of Images and Its Impact on Learning

by   Phillip Pope, et al.

It is widely believed that natural image data exhibits low-dimensional structure despite the high dimensionality of conventional pixel representations. This idea underlies a common intuition for the remarkable success of deep learning in computer vision. In this work, we apply dimension estimation tools to popular datasets and investigate the role of low-dimensional structure in deep learning. We find that common natural image datasets indeed have very low intrinsic dimension relative to the high number of pixels in the images. Additionally, we find that low dimensional datasets are easier for neural networks to learn, and models solving these tasks generalize better from training to test data. Along the way, we develop a technique for validating our dimension estimation tools on synthetic data generated by GANs allowing us to actively manipulate the intrinsic dimension by controlling the image generation process. Code for our experiments may be found here


Adaptive Approximation and Estimation of Deep Neural Network to Intrinsic Dimensionality

We theoretically prove that the generalization performance of deep neura...

The Intrinsic Manifolds of Radiological Images and their Role in Deep Learning

The manifold hypothesis is a core mechanism behind the success of deep l...

Intrinsic Dimension Estimation

It has long been thought that high-dimensional data encountered in many ...

Dimensionality compression and expansion in Deep Neural Networks

Datasets such as images, text, or movies are embedded in high-dimensiona...

Intrinsic dimension estimation for discrete metrics

Real world-datasets characterized by discrete features are ubiquitous: f...

A Theoretical Analysis of Deep Neural Networks for Texture Classification

We investigate the use of Deep Neural Networks for the classification of...

GENHOP: An Image Generation Method Based on Successive Subspace Learning

Being different from deep-learning-based (DL-based) image generation met...