Large-Margin Classification with Multiple Decision Rules

by   Patrick K. Kimes, et al.
University of North Carolina at Chapel Hill

Binary classification is a common statistical learning problem in which a model is estimated on a set of covariates for some outcome indicating the membership of one of two classes. In the literature, there exists a distinction between hard and soft classification. In soft classification, the conditional class probability is modeled as a function of the covariates. In contrast, hard classification methods only target the optimal prediction boundary. While hard and soft classification methods have been studied extensively, not much work has been done to compare the actual tasks of hard and soft classification. In this paper we propose a spectrum of statistical learning problems which span the hard and soft classification tasks based on fitting multiple decision rules to the data. By doing so, we reveal a novel collection of learning tasks of increasing complexity. We study the problems using the framework of large-margin classifiers and a class of piecewise linear convex surrogates, for which we derive statistical properties and a corresponding sub-gradient descent algorithm. We conclude by applying our approach to simulation settings and a magnetic resonance imaging (MRI) dataset from the Alzheimer's Disease Neuroimaging Initiative (ADNI) study.


page 8

page 27


Horocycle Decision Boundaries for Large Margin Classification in Hyperbolic Space

Hyperbolic spaces have been quite popular in the recent past for represe...

Soft Rule Ensembles for Statistical Learning

In this article supervised learning problems are solved using soft rule ...

Hybrid Model using Feature Extraction and Non-linear SVM for Brain Tumor Classification

It is essential to classify brain tumors from magnetic resonance imaging...

Feature Learning and Classification in Neuroimaging: Predicting Cognitive Impairment from Magnetic Resonance Imaging

Due to the rapid innovation of technology and the desire to find and emp...

Soft-margin classification of object manifolds

A neural population responding to multiple appearances of a single objec...

SpineOne: A One-Stage Detection Framework for Degenerative Discs and Vertebrae

Spinal degeneration plagues many elders, office workers, and even the yo...

A Novel ECOC Algorithm with Centroid Distance Based Soft Coding Scheme

In ECOC framework, the ternary coding strategy is widely deployed in cod...

Please sign up or login with your details

Forgot password? Click here to reset