Feature Importance Ranking for Deep Learning

by   Maksymilian Wojtas, et al.

Feature importance ranking has become a powerful tool for explainable AI. However, its nature of combinatorial optimization poses a great challenge for deep learning. In this paper, we propose a novel dual-net architecture consisting of operator and selector for discovery of an optimal feature subset of a fixed size and ranking the importance of those features in the optimal subset simultaneously. During learning, the operator is trained for a supervised learning task via optimal feature subset candidates generated by the selector that learns predicting the learning performance of the operator working on different optimal subset candidates. We develop an alternate learning algorithm that trains two nets jointly and incorporates a stochastic local search procedure into learning to address the combinatorial optimization challenge. In deployment, the selector generates an optimal feature subset and ranks feature importance, while the operator makes predictions based on the optimal subset for test data. A thorough evaluation on synthetic, benchmark and real data sets suggests that our approach outperforms several state-of-the-art feature importance ranking and supervised feature selection methods. (Our source code is available: https://github.com/maksym33/FeatureImportanceDL)


page 6

page 7

page 12

page 19

page 20

page 23

page 24


Dynamic Partial Removal: A Neural Network Heuristic for Large Neighborhood Search

This paper presents a novel neural network design that learns the heuris...

ML4CO-KIDA: Knowledge Inheritance in Data Aggregation

The Machine Learning for Combinatorial Optimization (ML4CO) NeurIPS 2021...

Elastic Net based Feature Ranking and Selection

Feature selection is important in data representation and intelligent di...

A Mention-Ranking Model for Abstract Anaphora Resolution

Resolving abstract anaphora is an important, but difficult task for text...

Accelerating E-Commerce Search Engine Ranking by Contextual Factor Selection

In industrial large-scale search systems, such as Taobao.com search for ...

A Study and Analysis of a Feature Subset Selection Technique using Penguin Search Optimization Algorithm (FS-PeSOA)

In today world of enormous amounts of data, it is very important to extr...

Domain Decorrelation with Potential Energy Ranking

Machine learning systems, especially the methods based on deep learning,...

Code Repositories