Log In Sign Up

Pareto GAN: Extending the Representational Power of GANs to Heavy-Tailed Distributions

by   Todd Huster, et al.

Generative adversarial networks (GANs) are often billed as "universal distribution learners", but precisely what distributions they can represent and learn is still an open question. Heavy-tailed distributions are prevalent in many different domains such as financial risk-assessment, physics, and epidemiology. We observe that existing GAN architectures do a poor job of matching the asymptotic behavior of heavy-tailed distributions, a problem that we show stems from their construction. Additionally, when faced with the infinite moments and large distances between outlier points that are characteristic of heavy-tailed distributions, common loss functions produce unstable or near-zero gradients. We address these problems with the Pareto GAN. A Pareto GAN leverages extreme value theory and the functional properties of neural networks to learn a distribution that matches the asymptotic behavior of the marginal distributions of the features. We identify issues with standard loss functions and propose the use of alternative metric spaces that enable stable and efficient learning. Finally, we evaluate our proposed approach on a variety of heavy-tailed datasets.


page 1

page 2

page 3

page 4


On some theoretical limitations of Generative Adversarial Networks

Generative Adversarial Networks have become a core technique in Machine ...

(f,Γ)-Divergences: Interpolating between f-Divergences and Integral Probability Metrics

We develop a general framework for constructing new information-theoreti...

Optimal Randomness in Swarm-based Search

Swarm-based search has been a hot topic for a long time. Among all the p...

Tails of Triangular Flows

Triangular maps are a construct in probability theory that allows the tr...

Beware the Gini Index! A New Inequality Measure

The Gini index underestimates inequality for heavy-tailed distributions:...

Independent Approximates enable closed-form estimation of heavy-tailed distributions

Independent Approximates (IAs) are proven to enable a closed-form estima...

Keep It Real: Tail Probabilities of Compound Heavy-Tailed Distributions

We propose an analytical approach to the computation of tail probabiliti...