Quit When You Can: Efficient Evaluation of Ensembles with Ordering Optimization

by   Serena Wang, et al.

Given a classifier ensemble and a set of examples to be classified, many examples may be confidently and accurately classified after only a subset of the base models in the ensemble are evaluated. This can reduce both mean latency and CPU while maintaining the high accuracy of the original ensemble. To achieve such gains, we propose jointly optimizing a fixed evaluation order of the base models and early-stopping thresholds. Our proposed objective is a combinatorial optimization problem, but we provide a greedy algorithm that achieves a 4-approximation of the optimal solution for certain cases. For those cases, this is also the best achievable polynomial time approximation bound unless P = NP. Experiments on benchmark and real-world problems show that the proposed Quit When You Can (QWYC) algorithm can speed-up average evaluation time by 2x--4x, and is around 1.5x faster than prior work. QWYC's joint optimization of ordering and thresholds also performed better in experiments than various fixed orderings, including gradient boosted trees' ordering.


page 1

page 2

page 3

page 4


Determinantal Point Processes Stochastic Approximation for Combinatorial Optimization

We study the problem of optimal subset selection from a set of correlate...

Enhancing Certifiable Robustness via a Deep Model Ensemble

We propose an algorithm to enhance certified robustness of a deep model ...

InfiniteBoost: building infinite ensembles with gradient descent

In machine learning ensemble methods have demonstrated high accuracy for...

Efficient PTAS for the Maximum Traveling Salesman Problem in a Metric Space of Fixed Doubling Dimension

The maximum traveling salesman problem (Max TSP) is one of the intensive...

Pruning variable selection ensembles

In the context of variable selection, ensemble learning has gained incre...

Towards Inference Efficient Deep Ensemble Learning

Ensemble methods can deliver surprising performance gains but also bring...

Approximating Biobjective Minimization Problems Using General Ordering Cones

This article investigates the approximation quality achievable for biobj...

Please sign up or login with your details

Forgot password? Click here to reset