
MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning
In this paper, we propose a novel meta learning approach for automatic c...
read it

Recursive Binary Neural Network Learning Model with 2.28b/Weight Storage Requirement
This paper presents a storageefficient learning model titled Recursive ...
read it

Generating Neural Networks with Neural Networks
Hypernetworks are neural networks that transform a random input vector i...
read it

Network with SubNetworks
We introduce network with subnetwork, a neural network which their weig...
read it

Detecting AI Trojans Using Meta Neural Analysis
Machine learning models, especially neural networks (NNs), have achieved...
read it

Variationaware Binarized Memristive Networks
The quantization of weights to binary states in Deep Neural Networks (DN...
read it

Learned Optimizers that Scale and Generalize
Learning to learn has emerged as an important direction for achieving ar...
read it
Classifying the classifier: dissecting the weight space of neural networks
This paper presents an empirical study on the weights of neural networks, where we interpret each model as a point in a highdimensional space – the neural weight space. To explore the complex structure of this space, we sample from a diverse selection of training variations (dataset, optimization procedure, architecture, etc.) of neural network classifiers, and train a large number of models to represent the weight space. Then, we use a machine learning approach for analyzing and extracting information from this space. Most centrally, we train a number of novel deep metaclassifiers with the objective of classifying different properties of the training setup by identifying their footprints in the weight space. Thus, the metaclassifiers probe for patterns induced by hyperparameters, so that we can quantify how much, where, and when these are encoded through the optimization process. This provides a novel and complementary view for explainable AI, and we show how metaclassifiers can reveal a great deal of information about the training setup and optimization, by only considering a small subset of randomly selected consecutive weights. To promote further research on the weight space, we release the neural weight space (NWS) dataset – a collection of 320K weight snapshots from 16K individually trained deep neural networks.
READ FULL TEXT
Comments
There are no comments yet.