Feature Importance Ranking for Deep Learning

10/18/2020
by   Maksymilian Wojtas, et al.
123

Feature importance ranking has become a powerful tool for explainable AI. However, its nature of combinatorial optimization poses a great challenge for deep learning. In this paper, we propose a novel dual-net architecture consisting of operator and selector for discovery of an optimal feature subset of a fixed size and ranking the importance of those features in the optimal subset simultaneously. During learning, the operator is trained for a supervised learning task via optimal feature subset candidates generated by the selector that learns predicting the learning performance of the operator working on different optimal subset candidates. We develop an alternate learning algorithm that trains two nets jointly and incorporates a stochastic local search procedure into learning to address the combinatorial optimization challenge. In deployment, the selector generates an optimal feature subset and ranks feature importance, while the operator makes predictions based on the optimal subset for test data. A thorough evaluation on synthetic, benchmark and real data sets suggests that our approach outperforms several state-of-the-art feature importance ranking and supervised feature selection methods. (Our source code is available: https://github.com/maksym33/FeatureImportanceDL)

READ FULL TEXT

page 6

page 7

page 12

page 19

page 20

page 23

page 24

research
05/19/2020

Dynamic Partial Removal: A Neural Network Heuristic for Large Neighborhood Search

This paper presents a novel neural network design that learns the heuris...
research
09/28/2022

Variance Tolerance Factors For Interpreting Neural Networks

Black box models only provide results for deep learning tasks and lack i...
research
01/25/2022

ML4CO-KIDA: Knowledge Inheritance in Data Aggregation

The Machine Learning for Combinatorial Optimization (ML4CO) NeurIPS 2021...
research
06/07/2017

A Mention-Ranking Model for Abstract Anaphora Resolution

Resolving abstract anaphora is an important, but difficult task for text...
research
03/02/2018

Accelerating E-Commerce Search Engine Ranking by Contextual Factor Selection

In industrial large-scale search systems, such as Taobao.com search for ...
research
06/15/2023

Ranking and Selection in Large-Scale Inference of Heteroscedastic Units

The allocation of limited resources to a large number of potential candi...
research
06/25/2023

Fast Classification with Sequential Feature Selection in Test Phase

This paper introduces a novel approach to active feature acquisition for...

Please sign up or login with your details

Forgot password? Click here to reset