DeepAI AI Chat
Log In Sign Up

Improving Simple Models with Confidence Profiles

by   Amit Dhurandhar, et al.

In this paper, we propose a new method called ProfWeight for transferring information from a pre-trained deep neural network that has a high test accuracy to a simpler interpretable model or a very shallow network of low complexity and a priori low test accuracy. We are motivated by applications in interpretability and model deployment in severely memory constrained environments (like sensors). Our method uses linear probes to generate confidence scores through flattened intermediate representations. Our transfer method involves a theoretically justified weighting of samples during the training of the simple model using confidence scores of these intermediate layers. The value of our method is first demonstrated on CIFAR-10, where our weighting method significantly improves (3-4 the number of Resnet blocks of a complex Resnet model. We further demonstrate operationally significant results on a real manufacturing problem, where we dramatically increase the test accuracy of a CART model (the domain standard) by roughly 13


page 1

page 2

page 3

page 4


Leveraging Simple Model Predictions for Enhancing its Performance

There has been recent interest in improving performance of simple models...

Why Out-of-distribution Detection in CNNs Does Not Like Mahalanobis – and What to Use Instead

Convolutional neural networks applied for real-world classification task...

Confidence Scoring Using Whitebox Meta-models with Linear Classifier Probes

We propose a confidence scoring mechanism for multi-layer neural network...

Calibrating Deep Neural Networks using Explicit Regularisation and Dynamic Data Pruning

Deep neural networks (DNN) are prone to miscalibrated predictions, often...

Understanding intermediate layers using linear classifier probes

Neural network models have a reputation for being black boxes. We propos...

On Modeling ASR Word Confidence

We present a new method for computing ASR word confidences that effectiv...

Distilling BERT for low complexity network training

This paper studies the efficiency of transferring BERT learnings to low ...