DeepAI AI Chat
Log In Sign Up

Neural Greedy Pursuit for Feature Selection

by   Sandipan Das, et al.
Weizmann Institute of Science
KTH Royal Institute of Technology

We propose a greedy algorithm to select N important features among P input features for a non-linear prediction problem. The features are selected one by one sequentially, in an iterative loss minimization procedure. We use neural networks as predictors in the algorithm to compute the loss and hence, we refer to our method as neural greedy pursuit (NGP). NGP is efficient in selecting N features when N ≪ P, and it provides a notion of feature importance in a descending order following the sequential selection procedure. We experimentally show that NGP provides better performance than several feature selection methods such as DeepLIFT and Drop-one-out loss. In addition, we experimentally show a phase transition behavior in which perfect selection of all N features without false positives is possible when the training data size exceeds a threshold.


page 2

page 6


Sequential Attention for Feature Selection

Feature selection is the problem of selecting a subset of features for a...

Estimation with Fast Landmark Selection in Robot Visual Navigation

We consider the visual feature selection to improve the estimation quali...

Controlling Costs: Feature Selection on a Budget

The traditional framework for feature selection treats all features as c...

Using Feature Weights to Improve Performance of Neural Networks

Different features have different relevance to a particular learning pro...

A study of local optima for learning feature interactions using neural networks

In many fields such as bioinformatics, high energy physics, power distri...

Online Selection of Diverse Committees

Citizens' assemblies need to represent subpopulations according to their...

Fast Feature Selection with Fairness Constraints

We study the fundamental problem of selecting optimal features for model...