Leveraging The Topological Consistencies of Learning in Deep Neural Networks

by   Stuart Synakowski, et al.

Recently, methods have been developed to accurately predict the testing performance of a Deep Neural Network (DNN) on a particular task, given statistics of its underlying topological structure. However, further leveraging this newly found insight for practical applications is intractable due to the high computational cost in terms of time and memory. In this work, we define a new class of topological features that accurately characterize the progress of learning while being quick to compute during running time. Additionally, our proposed topological features are readily equipped for backpropagation, meaning that they can be incorporated in end-to-end training. Our newly developed practical topological characterization of DNNs allows for an additional set of applications. We first show we can predict the performance of a DNN without a testing set and without the need for high-performance computing. We also demonstrate our topological characterization of DNNs is effective in estimating task similarity. Lastly, we show we can induce learning in DNNs by actively constraining the DNN's topological structure. This opens up new avenues in constricting the underlying structure of DNNs in a meta-learning framework.


page 1

page 7

page 10

page 11

page 14

page 18


Topological Structure is Predictive of Deep Neural Network Success in Learning

Machine learning has become a fundamental tool in modern science, yet it...

Analysis on the Nonlinear Dynamics of Deep Neural Networks: Topological Entropy and Chaos

The theoretical explanation for deep neural network (DNN) is still an op...

Learning Abstract Task Representations

A proper form of data characterization can guide the process of learning...

Stacking-Based Deep Neural Network: Deep Analytic Network for Pattern Classification

Stacking-based deep neural network (S-DNN) is aggregated with pluralitie...

Skyline: Interactive In-Editor Computational Performance Profiling for Deep Neural Network Training

Training a state-of-the-art deep neural network (DNN) is a computational...

Please sign up or login with your details

Forgot password? Click here to reset