DeepAI
Log In Sign Up

Nonlinear Monte Carlo Method for Imbalanced Data Learning

10/27/2020
by   Xuli Shen, et al.
0

For basic machine learning problems, expected error is used to evaluate model performance. Since the distribution of data is usually unknown, we can make simple hypothesis that the data are sampled independently and identically distributed (i.i.d.) and the mean value of loss function is used as the empirical risk by Law of Large Numbers (LLN). This is known as the Monte Carlo method. However, when LLN is not applicable, such as imbalanced data problems, empirical risk will cause overfitting and might decrease robustness and generalization ability. Inspired by the framework of nonlinear expectation theory, we substitute the mean value of loss function with the maximum value of subgroup mean loss. We call it nonlinear Monte Carlo method. In order to use numerical method of optimization, we linearize and smooth the functional of maximum empirical risk and get the descent direction via quadratic programming. With the proposed method, we achieve better performance than SOTA backbone models with less training steps, and more robustness for basic regression and imbalanced classification tasks.

READ FULL TEXT
08/21/2018

Wrapped Loss Function for Regularizing Nonconforming Residual Distributions

Multi-output is essential in machine learning that it might suffer from ...
08/20/2019

Sensitivity estimation of conditional value at risk using randomized quasi-Monte Carlo

Conditional value at risk (CVaR) is a popular measure for quantifying po...
05/24/2018

Strategic Monte Carlo Methods for State and Parameter Estimation in High Dimensional Nonlinear Problems

In statistical data assimilation one seeks the largest maximum of the co...
11/14/2022

Regression-based Monte Carlo Integration

Monte Carlo integration is typically interpreted as an estimator of the ...
05/21/2018

Adaptive Monte-Carlo Optimization

The celebrated Monte Carlo method estimates a quantity that is expensive...
06/25/2019

Monte Carlo Gradient Estimation in Machine Learning

This paper is a broad and accessible survey of the methods we have at ou...
06/20/2022

Model Optimization in Imbalanced Regression

Imbalanced domain learning aims to produce accurate models in predicting...