DeepAI AI Chat
Log In Sign Up

Binary Stochastic Filtering: a Solution for Supervised Feature Selection and Neural Network Shape Optimization

by   Andrii Trelin, et al.

Binary Stochastic Filtering (BSF), the algorithm for feature selection and neuron pruning is proposed in this work. Filtering layer stochastically passes or filters out features based on individual weights, which are tuned during neural network training process. By placing BSF after the neural network input, the filtering of input features is performed, i.e. feature selection. More then 5-fold dimensionality decrease was achieved in the experiments. Placing BSF layer in between hidden layers allows filtering of neuron outputs and could be used for neuron pruning. Up to 34-fold decrease in the number of weights in the network was reached, which corresponds to the significant increase of performance, that is especially important for mobile and embedded applications.


Binary Stochastic Filtering: feature selection and beyond

Feature selection is one of the most decisive tools in understanding dat...

Feature Selection Based on Sparse Neural Network Layer with Normalizing Constraints

Feature selection is important step in machine learning since it has sho...

LassoLayer: Nonlinear Feature Selection by Switching One-to-one Links

Along with the desire to address more complex problems, feature selectio...

NISP: Pruning Networks using Neuron Importance Score Propagation

To reduce the significant redundancy in deep Convolutional Neural Networ...

Neuron Merging: Compensating for Pruned Neurons

Network pruning is widely used to lighten and accelerate neural network ...

Deep supervised feature selection using Stochastic Gates

In this study, we propose a novel non-parametric embedded feature select...

Neural Response Interpretation through the Lens of Critical Pathways

Is critical input information encoded in specific sparse pathways within...