Accurate Small Models using Adaptive Sampling

10/08/2022
by   Abhishek Ghose, et al.
0

We highlight the utility of a certain property of model training: instead of drawing training data from the same distribution as test data, learning a different training distribution often improves accuracy, especially at small model sizes. This provides a way to build accurate small models, which are attractive for interpretability and resource-constrained environments. Here we empirically show that this principle is both general and effective: it may be used across tasks/model families, and it can augment prediction accuracy of traditional models to the extent they are competitive with specialized techniques. The tasks we consider are explainable clustering and prototype-based classification. We also look at Random Forests to illustrate how this principle may be applied to accommodate multiple size constraints, e.g., number of trees and maximum depth per tree. Results using multiple datasets are presented and are shown to be statistically significant.

READ FULL TEXT

page 4

page 6

page 7

page 11

page 12

page 13

research
06/17/2019

Learning Interpretable Models Using an Oracle

As Machine Learning (ML) becomes pervasive in various real world systems...
research
07/05/2022

An Approximation Method for Fitted Random Forests

Random Forests (RF) is a popular machine learning method for classificat...
research
10/07/2022

ProGReST: Prototypical Graph Regression Soft Trees for Molecular Property Prediction

In this work, we propose the novel Prototypical Graph Regression Self-ex...
research
05/04/2019

Optimal Resampling for Learning Small Models

Models often need to be constrained to a certain size for them to be con...
research
09/26/2022

Knowledge Distillation to Ensemble Global and Interpretable Prototype-Based Mammogram Classification Models

State-of-the-art (SOTA) deep learning mammogram classifiers, trained wit...
research
08/27/2019

Locally Optimized Random Forests

Standard supervised learning procedures are validated against a test set...
research
12/22/2017

Inverse Classification for Comparison-based Interpretability in Machine Learning

In the context of post-hoc interpretability, this paper addresses the ta...

Please sign up or login with your details

Forgot password? Click here to reset