DeepAI AI Chat
Log In Sign Up

Characterizing information loss in a chaotic double pendulum with the Information Bottleneck

by   Kieran A. Murphy, et al.

A hallmark of chaotic dynamics is the loss of information with time. Although information loss is often expressed through a connection to Lyapunov exponents – valid in the limit of high information about the system state – this picture misses the rich spectrum of information decay across different levels of granularity. Here we show how machine learning presents new opportunities for the study of information loss in chaotic dynamics, with a double pendulum serving as a model system. We use the Information Bottleneck as a training objective for a neural network to extract information from the state of the system that is optimally predictive of the future state after a prescribed time horizon. We then decompose the optimally predictive information by distributing a bottleneck to each state variable, recovering the relative importance of the variables in determining future evolution. The framework we develop is broadly applicable to chaotic systems and pragmatic to apply, leveraging data and machine learning to monitor the limits of predictability and map out the loss of information.


page 3

page 4


Machine Learning using the Variational Predictive Information Bottleneck with a Validation Set

Zellner (1988) modeled statistical inference in terms of information pro...

Interpretability with full complexity by constraining feature information

Interpretability is a pressing issue for machine learning. Common approa...

Learning Optimal Representations with the Decodable Information Bottleneck

We address the question of characterizing and finding optimal representa...

Information bottleneck theory of high-dimensional regression: relevancy, efficiency and optimality

Avoiding overfitting is a central challenge in machine learning, yet man...

Transfer Entropy Bottleneck: Learning Sequence to Sequence Information Transfer

When presented with a data stream of two statistically dependent variabl...

Multivariate Information Bottleneck

The Information bottleneck method is an unsupervised non-parametric data...

Extracting Global Dynamics of Loss Landscape in Deep Learning Models

Deep learning models evolve through training to learn the manifold in wh...