DeepAI AI Chat
Log In Sign Up

Learning with Fenchel-Young Losses

by   Mathieu Blondel, et al.
Mathieu Blondel
Unbabel Inc.

Over the past decades, numerous loss functions have been been proposed for a variety of supervised learning tasks, including regression, classification, ranking, and more generally structured prediction. Understanding the core principles and theoretical properties underpinning these losses is key to choose the right loss for the right problem, as well as to create new losses which combine their strengths. In this paper, we introduce Fenchel-Young losses, a generic way to construct a convex loss function for a regularized prediction function. We provide an in-depth study of their properties in a very broad setting, covering all the aforementioned supervised learning tasks, and revealing new connections between sparsity, generalized entropies, and separation margins. We show that Fenchel-Young losses unify many well-known loss functions and allow to create useful new ones easily. Finally, we derive efficient predictive and training algorithms, making Fenchel-Young losses appealing both in theory and practice.


page 1

page 2

page 3

page 4


Learning Classifiers with Fenchel-Young Losses: Generalized Entropies, Margins, and Algorithms

We study in this paper Fenchel-Young losses, a generic way to construct ...

Learning Energy Networks with Generalized Fenchel-Young Losses

Energy-based models, a.k.a. energy networks, perform inference by optimi...

Structured Prediction with Projection Oracles

We propose in this paper a general framework for deriving loss functions...

Supervised Learning: No Loss No Cry

Supervised learning requires the specification of a loss function to min...

All your loss are belong to Bayes

Loss functions are a cornerstone of machine learning and the starting po...

Learning State-Dependent Losses for Inverse Dynamics Learning

Being able to quickly adapt to changes in dynamics is paramount in model...

LegendreTron: Uprising Proper Multiclass Loss Learning

Loss functions serve as the foundation of supervised learning and are of...

Code Repositories


Probabilistic classification in PyTorch/TensorFlow/scikit-learn with Fenchel-Young losses

view repo