DeepAI AI Chat
Log In Sign Up

Density Fixing: Simple yet Effective Regularization Method based on the Class Prior

by   Masanari Kimura, et al.

Machine learning models suffer from overfitting, which is caused by a lack of labeled data. To tackle this problem, we proposed a framework of regularization methods, called density-fixing, that can be used commonly for supervised and semi-supervised learning. Our proposed regularization method improves the generalization performance by forcing the model to approximate the class's prior distribution or the frequency of occurrence. This regularization term is naturally derived from the formula of maximum likelihood estimation and is theoretically justified. We further investigated the asymptotic behavior of the proposed method and how the regularization terms behave when assuming a prior distribution of several classes in practice. Experimental results on multiple benchmark datasets are sufficient to support our argument, and we suggest that this simple and effective regularization method is useful in real-world machine learning problems.


page 1

page 2

page 3

page 4


Dense FixMatch: a simple semi-supervised learning method for pixel-wise prediction tasks

We propose Dense FixMatch, a simple method for online semi-supervised le...

Semi-Supervised RF Fingerprinting with Consistency-Based Regularization

As a promising non-password authentication technology, radio frequency (...

Mixup Training as the Complexity Reduction

Machine learning has achieved remarkable results in recent years due to ...

Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning

Effective convolutional neural networks are trained on large sets of lab...

Improving Robustness of Attention Models on Graphs

Machine learning models that can exploit the inherent structure in data ...

Revisiting Consistency Regularization for Semi-Supervised Learning

Consistency regularization is one of the most widely-used techniques for...

Human Activity Recognition Using Robust Adaptive Privileged Probabilistic Learning

In this work, a novel method based on the learning using privileged info...