Training Effective Ensemble on Imbalanced Data by Self-paced Harmonizing Classification Hardness

09/08/2019
by   Zhining Liu, et al.
0

Many real-world applications reveal difficulties in learning classifiers from imbalanced data. The rising big data era has been witnessing more classification tasks with large-scale but extremely imbalance and low-quality datasets. Most of existing learning methods suffer from poor performance or low computation efficiency under such a scenario. To tackle this problem, we conduct deep investigations into the nature of class imbalance, which reveals that not only the disproportion between classes, but also other difficulties embedded in the nature of data, especially, noises and class overlapping, prevent us from learning effective classifiers. Taking those factors into consideration, we propose a novel framework for imbalance classification that aims to generate a strong ensemble by self-paced harmonizing data hardness via under-sampling. Extensive experiments have shown that this new framework, while being very computationally efficient, can lead to robust performance even under highly overlapping classes and extremely skewed distribution. Note that, our methods can be easily adapted to most of existing learning methods (e.g., C4.5, SVM, GBDT and Neural Network) to boost their performance on imbalanced data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/08/2019

Self-paced Ensemble for Highly Imbalanced Massive Data Classification

Many real-world applications reveal difficulties in learning classifiers...
research
04/09/2019

Classification of Imbalanced Data with a Geometric Digraph Family

We use a geometric digraph family called class cover catch digraphs (CCC...
research
08/20/2022

A Novel Hybrid Sampling Framework for Imbalanced Learning

Class imbalance is a frequently occurring scenario in classification tas...
research
11/15/2017

LIUBoost : Locality Informed Underboosting for Imbalanced Data Classification

The problem of class imbalance along with class-overlapping has become a...
research
04/26/2020

Classification Trees for Imbalanced and Sparse Data: Surface-to-Volume Regularization

Classification algorithms face difficulties when one or more classes hav...
research
12/02/2019

Matrix sketching for supervised classification with imbalanced classes

Matrix sketching is a recently developed data compression technique. An ...
research
03/21/2022

ASE: Anomaly Scoring Based Ensemble Learning for Imbalanced Datasets

Nowadays, many industries have applied classification algorithms to help...

Please sign up or login with your details

Forgot password? Click here to reset