Learning with rare data: Using active importance sampling to optimize objectives dominated by rare events

by   Grant M. Rotskoff, et al.

Deep neural networks, when optimized with sufficient data, provide accurate representations of high-dimensional functions; in contrast, function approximation techniques that have predominated in scientific computing do not scale well with dimensionality. As a result, many high-dimensional sampling and approximation problems once thought intractable are being revisited through the lens of machine learning. While the promise of unparalleled accuracy may suggest a renaissance for applications that require parameterizing representations of complex systems, in many applications gathering sufficient data to develop such a representation remains a significant challenge. Here we introduce an approach that combines rare events sampling techniques with neural network optimization to optimize objective functions that are dominated by rare events. We show that importance sampling reduces the asymptotic variance of the solution to a learning problem, suggesting benefits for generalization. We study our algorithm in the context of learning dynamical transition pathways between two states of a system, a problem with applications in statistical physics and implications in machine learning theory. Our numerical experiments demonstrate that we can successfully learn even with the compounding difficulties of high-dimension and rare data.


Computing Committor Functions for the Study of Rare Events Using Deep Learning

The committor function is a central object of study in understanding tra...

Large deviation theory-based adaptive importance sampling for rare events in high dimensions

We propose a method for the accurate estimation of rare event or failure...

SYMPAIS: SYMbolic Parallel Adaptive Importance Sampling for Probabilistic Program Analysis

Probabilistic software analysis aims at quantifying the probability of a...

Inexact iterative numerical linear algebra for neural network-based spectral estimation and rare-event prediction

Understanding dynamics in complex systems is challenging because there a...

Deep importance sampling using tensor-trains with application to a priori and a posteriori rare event estimation

We propose a deep importance sampling method that is suitable for estima...

Rare-Event Simulation for Neural Network and Random Forest Predictors

We study rare-event simulation for a class of problems where the target ...

Test Against High-Dimensional Uncertainties: Accelerated Evaluation of Autonomous Vehicles with Deep Importance Sampling

Evaluating the performance of autonomous vehicles (AV) and their complex...

Please sign up or login with your details

Forgot password? Click here to reset