Max-Margin Feature Selection

06/14/2016
by   Yamuna Prasad, et al.
0

Many machine learning applications such as in vision, biology and social networking deal with data in high dimensions. Feature selection is typically employed to select a subset of features which im- proves generalization accuracy as well as reduces the computational cost of learning the model. One of the criteria used for feature selection is to jointly minimize the redundancy and maximize the rele- vance of the selected features. In this paper, we formulate the task of feature selection as a one class SVM problem in a space where features correspond to the data points and instances correspond to the dimensions. The goal is to look for a representative subset of the features (support vectors) which describes the boundary for the region where the set of the features (data points) exists. This leads to a joint optimization of relevance and redundancy in a principled max-margin framework. Additionally, our formulation enables us to leverage existing techniques for optimizing the SVM objective resulting in highly computationally efficient solutions for the task of feature selection. Specifically, we employ the dual coordinate descent algorithm (Hsieh et al., 2008), originally proposed for SVMs, for our formulation. We use a sparse representation to deal with data in very high dimensions. Experiments on seven publicly available benchmark datasets from a variety of domains show that our approach results in orders of magnitude faster solutions even while retaining the same level of accuracy compared to the state of the art feature selection techniques.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/16/2021

Effective Streaming Evolutionary Feature Selection Using Dynamic Optimization

Feature selection is a key issue in machine learning and data mining. A ...
research
04/21/2020

A novel embedded min-max approach for feature selection in nonlinear SVM classification

In recent years, feature selection has become a challenging problem in s...
research
10/16/2020

Feature Selection for Huge Data via Minipatch Learning

Feature selection often leads to increased model interpretability, faste...
research
08/15/2019

Maximum Relevance and Minimum Redundancy Feature Selection Methods for a Marketing Machine Learning Platform

In machine learning applications for online product offerings and market...
research
07/27/2023

MVMR-FS : Non-parametric feature selection algorithm based on Maximum inter-class Variation and Minimum Redundancy

How to accurately measure the relevance and redundancy of features is an...
research
05/28/2019

Efficient Wrapper Feature Selection using Autoencoder and Model Based Elimination

We propose a computationally efficient wrapper feature selection method ...
research
11/27/2014

Features in Concert: Discriminative Feature Selection meets Unsupervised Clustering

Feature selection is an essential problem in computer vision, important ...

Please sign up or login with your details

Forgot password? Click here to reset