DeepAI AI Chat
Log In Sign Up

NGBoost: Natural Gradient Boosting for Probabilistic Prediction

by   Tony Duan, et al.

We present Natural Gradient Boosting (NGBoost), an algorithm which brings probabilistic prediction capability to gradient boosting in a generic way. Predictive uncertainty estimation is crucial in many applications such as healthcare and weather forecasting. Probabilistic prediction, which is the approach where the model outputs a full probability distribution over the entire outcome space, is a natural way to quantify those uncertainties. Gradient Boosting Machines have been widely successful in prediction tasks on structured input data, but a simple boosting solution for probabilistic prediction of real valued outputs is yet to be made. NGBoost is a gradient boosting approach which uses the Natural Gradient to address technical challenges that makes generic probabilistic prediction hard with existing gradient boosting methods. Our approach is modular with respect to the choice of base learner, probability distribution, and scoring rule. We show empirically on several regression datasets that NGBoost provides competitive predictive performance of both uncertainty estimates and traditional metrics.


Multivariate Probabilistic Regression with Natural Gradient Boosting

Many single-target regression problems require estimates of uncertainty ...

Uncertainty in Gradient Boosting via Ensembles

Gradient boosting is a powerful machine learning technique that is parti...

RoNGBa: A Robustly Optimized Natural Gradient Boosting Training Approach with Leaf Number Clipping

Natural gradient has been recently introduced to the field of boosting t...

Large-scale Uncertainty Estimation and Its Application in Revenue Forecast of SMEs

The economic and banking importance of the small and medium enterprise (...

Probabilistic Models for Manufacturing Lead Times

In this study, we utilize Gaussian processes, probabilistic neural netwo...

A Bagging and Boosting Based Convexly Combined Optimum Mixture Probabilistic Model

Unlike previous studies on mixture distributions, a bagging and boosting...

VAT tax gap prediction: a 2-steps Gradient Boosting approach

Tax evasion is the illegal non-payment of taxes by individuals, corporat...

Code Repositories


Natural Gradient Boosting for Probabilistic Prediction

view repo


Simple NGBoost Implementation written by numpy and sklearn.

view repo


A CLI Tuner of NGBoost

view repo