Dynamic Instance-Wise Classification in Correlated Feature Spaces

In a typical supervised machine learning setting, the predictions on all test instances are based on a common subset of features discovered during model training. However, using a different subset of features that is most informative for each test instance individually may not only improve prediction accuracy, but also the overall interpretability of the model. At the same time, feature selection methods for classification have been known to be the most effective when many features are irrelevant and/or uncorrelated. In fact, feature selection ignoring correlations between features can lead to poor classification performance. In this work, a Bayesian network is utilized to model feature dependencies. Using the dependency network, a new method is proposed that sequentially selects the best feature to evaluate for each test instance individually, and stops the selection process to make a prediction once it determines that no further improvement can be achieved with respect to classification accuracy. The optimum number of features to acquire and the optimum classification strategy are derived for each test instance. The theoretical properties of the optimum solution are analyzed, and a new algorithm is proposed that takes advantage of these properties to implement a robust and scalable solution for high dimensional settings. The effectiveness, generalizability, and scalability of the proposed method is illustrated on a variety of real-world datasets from diverse application domains.

READ FULL TEXT

page 2

page 3

page 4

page 5

page 6

page 8

page 10

page 11

research
04/21/2020

On-the-Fly Joint Feature Selection and Classification

Joint feature selection and classification in an online setting is essen...
research
07/06/2022

DIWIFT: Discovering Instance-wise Influential Features for Tabular Data

Tabular data is one of the most common data storage formats in business ...
research
06/06/2017

Embedding Feature Selection for Large-scale Hierarchical Classification

Large-scale Hierarchical Classification (HC) involves datasets consistin...
research
06/27/2012

Discovering Support and Affiliated Features from Very High Dimensions

In this paper, a novel learning paradigm is presented to automatically i...
research
07/13/2019

A Study and Analysis of a Feature Subset Selection Technique using Penguin Search Optimization Algorithm (FS-PeSOA)

In today world of enormous amounts of data, it is very important to extr...
research
06/21/2023

ProtoGate: Prototype-based Neural Networks with Local Feature Selection for Tabular Biomedical Data

Tabular biomedical data poses challenges in machine learning because it ...
research
10/16/2021

An Evolutionary Correlation-aware Feature Selection Method for Classification Problems

The population-based optimization algorithms have provided promising res...

Please sign up or login with your details

Forgot password? Click here to reset