Implicit Generation and Generalization in Energy-Based Models

by   Yilun Du, et al.

Energy based models (EBMs) are appealing due to their generality and simplicity in likelihood modeling, but have been traditionally difficult to train. We present techniques to scale MCMC based EBM training, on continuous neural networks, and show its success on the high-dimensional data domains of ImageNet32x32, ImageNet128x128, CIFAR-10, and robotic hand trajectories, achieving significantly better samples than other likelihood models and on par with contemporary GAN approaches, while covering all modes of the data. We highlight unique capabilities of implicit generation, such as energy compositionality and corrupt image reconstruction and completion. Finally, we show that EBMs generalize well and are able to achieve state-of-the-art out-of-distribution classification, exhibit adversarially robust classification, coherent long term predicted trajectory roll-outs, and generate zero-shot compositions of models.


page 4

page 5

page 6

page 8

page 9

page 11

page 12

page 13


No MCMC for me: Amortized sampling for fast and stable training of energy-based models

Energy-Based Models (EBMs) present a flexible and appealing way to repre...

M-EBM: Towards Understanding the Manifolds of Energy-Based Models

Energy-based models (EBMs) exhibit a variety of desirable properties in ...

Versatile Energy-Based Models for High Energy Physics

Energy-based models have the natural advantage of flexibility in the for...

Conditional Energy-Based Models for Implicit Policies: The Gap between Theory and Practice

We present our findings in the gap between theory and practice of using ...

Controlling Steering with Energy-Based Models

So-called implicit behavioral cloning with energy-based models has shown...

Revisiting Energy Based Models as Policies: Ranking Noise Contrastive Estimation and Interpolating Energy Models

A crucial design decision for any robot learning pipeline is the choice ...

Energy-Based Processes for Exchangeable Data

Recently there has been growing interest in modeling sets with exchangea...

Code Repositories


Code for Implicit Generation and Generalization with Energy Based Models

view repo


Implicit Generation and Generalization in Energy Based Models in PyTorch

view repo


EBM image generation

view repo


An energy based model implemented in Pytorch and trained with multiple persistent markov chains.

view repo


Source code and pack for EBMs training. ✅ Adapted 𝚋𝚢 𝚋𝚝𝟹𝚐𝚕

view repo

Please sign up or login with your details

Forgot password? Click here to reset