DeepAI AI Chat
Log In Sign Up

A stochastic approach to handle knapsack problems in the creation of ensembles

by   Andras Hajdu, et al.

Ensemble-based methods are highly popular approaches that increase the accuracy of a decision by aggregating the opinions of individual voters. The common point is to maximize accuracy; however, a natural limitation occurs if incremental costs are also assigned to the individual voters. Consequently, we investigate creating ensembles under an additional constraint on the total cost of the members. This task can be formulated as a knapsack problem, where the energy is the ensemble accuracy formed by some aggregation rules. However, the generally applied aggregation rules lead to a nonseparable energy function, which takes the common solution tools – such as dynamic programming – out of action. We introduce a novel stochastic approach that considers the energy as the joint probability function of the member accuracies. This type of knowledge can be efficiently incorporated in a stochastic search process as a stopping rule, since we have the information on the expected accuracy or, alternatively, the probability of finding more accurate ensembles. Experimental analyses of the created ensembles of pattern classifiers and object detectors confirm the efficiency of our approach. Moreover, we propose a novel stochastic search strategy that better fits the energy, compared with general approaches such as simulated annealing.


Optimizing Majority Voting Based Systems Under a Resource Constraint for Multiclass Problems

Ensemble-based approaches are very effective in various fields in raisin...

Learning Locally Interpretable Rule Ensemble

This paper proposes a new framework for learning a rule ensemble model t...

Forming Ensembles at Runtime: A Machine Learning Approach

Smart system applications (SSAs) built on top of cyber-physical and soci...

Novel ensemble collaboration method for dynamic scheduling problems

Dynamic scheduling problems are important optimisation problems with man...

On Aggregation in Ensembles of Multilabel Classifiers

While a variety of ensemble methods for multilabel classification have b...

Neural network ensembles: Evaluation of aggregation algorithms

Ensembles of artificial neural networks show improved generalization cap...

New Probabilistic-Dynamic Multi-Method Ensembles for Optimization based on the CRO-SL

In this paper we propose new probabilistic and dynamic (adaptive) strate...