It Is Likely That Your Loss Should be a Likelihood

07/12/2020
by   Mark Hamilton, et al.
0

We recall that certain common losses are simplified likelihoods and instead argue for optimizing full likelihoods that include their parameters, such as the variance of the normal distribution and the temperature of the softmax distribution. Joint optimization of likelihood and model parameters can adaptively tune the scales and shapes of losses and the weights of regularizers. We survey and systematically evaluate how to parameterize and apply likelihood parameters for robust modeling and re-calibration. Additionally, we propose adaptively tuning L_2 and L_1 weights by fitting the scale parameters of normal and Laplace priors and introduce more flexible element-wise regularizers.

READ FULL TEXT

page 4

page 10

page 14

research
03/14/2018

Joint Modelling of Location, Scale and Skewness Parameters of the Skew Laplace Normal Distribution

In this article, we propose joint location, scale and skewness models of...
research
06/03/2022

Beta Generalized Normal Distribution with an Application for SAR Image Processing

We introduce the beta generalized normal distribution which is obtained ...
research
10/01/2019

The Balakrishnan Alpha Skew Laplace Distribution: Properties and Its Applications

In this study by considering Balakrishnan mechanism a new form of alpha ...
research
03/06/2017

On parameters transformations for emulating sparse priors using variational-Laplace inference

So-called sparse estimators arise in the context of model fitting, when ...
research
05/07/2019

P2SGrad: Refined Gradients for Optimizing Deep Face Models

Cosine-based softmax losses significantly improve the performance of dee...
research
03/10/2020

Learning State-Dependent Losses for Inverse Dynamics Learning

Being able to quickly adapt to changes in dynamics is paramount in model...
research
02/05/2019

Discovering bursts revisited: guaranteed optimization of the model parameters

One of the classic data mining tasks is to discover bursts, time interva...

Please sign up or login with your details

Forgot password? Click here to reset