DeepAI AI Chat
Log In Sign Up

Accelerated proximal boosting

by   Erwan Fouillen, et al.

Gradient boosting is a prediction method that iteratively combines weak learners to produce a complex and accurate model. From an optimization point of view, the learning procedure of gradient boosting mimics a gradient descent on a functional variable. This paper proposes to build upon the proximal point algorithm when the empirical risk to minimize is not differentiable. In addition, the novel boosting approach, called accelerated proximal boosting, benefits from Nesterov's acceleration in the same way as gradient boosting [Biau et al., 2018]. Advantages of leveraging proximal methods for boosting are illustrated by numerical experiments on simulated and real-world data. In particular, we exhibit a favorable comparison over gradient boosting regarding convergence rate and prediction accuracy.


page 10

page 13

page 15

page 17

page 18


Accelerated Componentwise Gradient Boosting using Efficient Data Representation and Momentum-based Optimization

Componentwise boosting (CWB), also known as model-based boosting, is a v...

An accelerated proximal gradient method for multiobjective optimization

Many descent methods for multiobjective optimization problems have been ...

Accelerated Gradient Boosting

Gradient tree boosting is a prediction algorithm that sequentially produ...

Accelerating Gradient Boosting Machine

Gradient Boosting Machine (GBM) is an extremely powerful supervised lear...

Functional Gradient Boosting based on Residual Network Perception

Residual Networks (ResNets) have become state-of-the-art models in deep ...

Boosting for Dynamical Systems

We propose a framework of boosting for learning and control in environme...

Distorted English Alphabet Identification : An application of Difference Boosting Algorithm

The difference-boosting algorithm is used on letters dataset from the UC...