Forest Representation Learning Guided by Margin Distribution

05/07/2019
by   Shen-Huan Lv, et al.
0

In this paper, we reformulate the forest representation learning approach as an additive model which boosts the augmented feature instead of the prediction. We substantially improve the upper bound of generalization gap from O(√(%s/%s) mm) to O( m/m), while λ - the margin ratio between the margin standard deviation and the margin mean is small enough. This tighter upper bound inspires us to optimize the margin distribution ratio λ. Therefore, we design the margin distribution reweighting approach (mdDF) to achieve small ratio λ by boosting the augmented feature. Experiments and visualizations confirm the effectiveness of the approach in terms of performance and representation learning ability. This study offers a novel understanding of the cascaded deep forest from the margin-theory perspective and further uses the mdDF approach to guide the layer-by-layer forest representation learning.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/27/2019

Margin-Based Generalization Lower Bounds for Boosted Classifiers

Boosting is one of the most successful ideas in machine learning. The mo...
research
10/09/2017

Function space analysis of deep learning representation layers

In this paper we propose a function space approach to Representation Lea...
research
04/15/2020

A Small Improvement to the Upper Bound on the Integrality Ratio for the s-t Path TSP

In this paper we investigate the integrality ratio of the standard LP re...
research
04/29/2022

On the Optimization of Margin Distribution

Margin has played an important role on the design and analysis of learni...
research
05/06/2022

The NT-Xent loss upper bound

Self-supervised learning is a growing paradigm in deep representation le...
research
06/11/2022

Learning Imbalanced Datasets with Maximum Margin Loss

A learning algorithm referred to as Maximum Margin (MM) is proposed for ...
research
10/16/2020

For self-supervised learning, Rationality implies generalization, provably

We prove a new upper bound on the generalization gap of classifiers that...

Please sign up or login with your details

Forgot password? Click here to reset