All your loss are belong to Bayes

by   Christian Walder, et al.

Loss functions are a cornerstone of machine learning and the starting point of most algorithms. Statistics and Bayesian decision theory have contributed, via properness, to elicit over the past decades a wide set of admissible losses in supervised learning, to which most popular choices belong (logistic, square, Matsushita, etc.). Rather than making a potentially biased ad hoc choice of the loss, there has recently been a boost in efforts to fit the loss to the domain at hand while training the model itself. The key approaches fit a canonical link, a function which monotonically relates the closed unit interval to R and can provide a proper loss via integration. In this paper, we rely on a broader view of proper composite losses and a recent construct from information geometry, source functions, whose fitting alleviates constraints faced by canonical links. We introduce a trick on squared Gaussian Processes to obtain a random process whose paths are compliant source functions with many desirable properties in the context of link estimation. Experimental results demonstrate substantial improvements over the state of the art.



There are no comments yet.


page 24

page 25


Supervised Learning: No Loss No Cry

Supervised learning requires the specification of a loss function to min...

Learning with Fenchel-Young Losses

Over the past decades, numerous loss functions have been been proposed f...

The Convexity and Design of Composite Multiclass Losses

We consider composite loss functions for multiclass prediction comprisin...

Proper-Composite Loss Functions in Arbitrary Dimensions

The study of a machine learning problem is in many ways is difficult to ...

Lower-bounded proper losses for weakly supervised classification

This paper discusses the problem of weakly supervised learning of classi...

Generalized Canonical Polyadic Tensor Decomposition

Tensor decomposition is a fundamental unsupervised machine learning meth...

f-GANs in an Information Geometric Nutshell

Nowozin et al showed last year how to extend the GAN principle to all f-...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.