Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One

12/06/2019
by   Will Grathwohl, et al.
19

We propose to reinterpret a standard discriminative classifier of p(y|x) as an energy based model for the joint distribution p(x,y). In this setting, the standard class probabilities can be easily computed as well as unnormalized values of p(x) and p(x|y). Within this framework, standard discriminative architectures may beused and the model can also be trained on unlabeled data. We demonstrate that energy based training of the joint distribution improves calibration, robustness, andout-of-distribution detection while also enabling our models to generate samplesrivaling the quality of recent GAN approaches. We improve upon recently proposed techniques for scaling up the training of energy based models and presentan approach which adds little overhead compared to standard classification training. Our approach is the first to achieve performance rivaling the state-of-the-artin both generative and discriminative learning within one hybrid model.

READ FULL TEXT

page 4

page 8

page 14

page 15

page 16

research
09/16/2022

Towards Bridging the Performance Gaps of Joint Energy-based Models

Can we train a hybrid discriminative-generative model within a single ne...
research
09/19/2021

JEM++: Improved Techniques for Training JEM

Joint Energy-based Model (JEM) is a recently proposed hybrid model that ...
research
03/07/2023

Stabilized training of joint energy-based models and their practical applications

The recently proposed Joint Energy-based Model (JEM) interprets discrimi...
research
11/01/2022

Consistent Training via Energy-Based GFlowNets for Modeling Discrete Joint Distributions

Generative Flow Networks (GFlowNets) have demonstrated significant perfo...
research
06/02/2023

On Feature Diversity in Energy-based Models

Energy-based learning is a powerful learning paradigm that encapsulates ...
research
04/03/2023

Non-Generative Energy Based Models

Energy-based models (EBM) have become increasingly popular within comput...
research
04/04/2023

EGC: Image Generation and Classification via a Single Energy-Based Model

Learning image classification and image generation using the same set of...

Please sign up or login with your details

Forgot password? Click here to reset