Life across the globe has survived and thrived by adapting to its local weather, including extreme events such as strong winds and floods from cyclones, drought and heat waves from blocking events and large-scale atmospheric oscillations, and critically-needed precipitation from atmospheric rivers. Driven by an ever-warming climate, extreme weather events are changing in frequency and intensity at an unprecedented pace [6, 29]. We need to understand these events and their driving mechanisms to enable communities to continue to adapt and thrive.
High-resolution, high-fidelity global climate models are an indispensable tool for investigating climate change. A multitude of climate change scenarios are now being simulated, each producing 100s of TBs of data. Currently, climate change is assessed in these simulations using summary statistics such as mean global sea surface temperature. This is inadequate for answering detailed questions about the effects of climate change on extreme weather events. Due to the sheer size and complexity of these simulated data sets, it is essential to develop robust and automated methods that can provide the deeper insights we seek.
Recently, supervised Deep Learning (DL) techniques have been applied to address this problem[17, 16, 3, 4]
. Further progress however has been stymied by two daunting challenges: reliance on labeled training data and interpretability of trained models. The DL models used in the above studies are trained using the automated heuristics of TECA for proximate labels. This is necessary because, simply put, there currently is no ground truth for pixel-level identification of extreme weather events . While the results in  show that DL can improve upon TECA, the results of 
reach accuracy rates over 97% and thus essentially just reproduce the output of TECA. The supervised learning paradigm of optimizing objective metrics (e.g. training and generalization error) breaks down here; TECA is not ground truth and we do not know how to train a DL model to disagree with TECA in just the right way to get closer to “ground truth”.
To avoid this issue, a campaign is currently underway to generate expert-labeled training data . Supervised DL models trained on this data will automate expert-level curation of large climate data sets for extreme weather detection. In this case there too will be challenges. Though an improvement over automated heuristics, expert-labeled data is still not an objective ground truth. Further, while human experts can debate the subtleties of physical characteristics of extreme weather events, the interpretability problem  prevents us from probing a trained DL model to determine exactly how and why it identifies (or misidentifies) specific events.
To circumvent these challenges of DL-based approaches, here we take an alternative physics-based unsupervised approach, complementary to DL.
Whereas DL takes inspiration from the human visual processing system to identify patterns in images without consideration for what constitutes a “pattern”, our method builds on a theory that seeks to understand the physical nature of pattern without consideration for the visual system that can readily identify such patterns. When viewing a video of a complex fluid flow we do not track the evolution of each individual pixel. Our vision instead focuses on relatively few collective features, generally referred to as coherent structures [15, 14], that the flow organizes around. Beyond fluid flows, coherent structures in spatiotemporal systems can similarly be understood as key organizing features that heavily dictate the dynamics of the full system, and thus provide a natural dimensionality reduction. Understanding the lower-dimensional coherent structures gets us most of the way to understanding and predicting the full higher-dimensional system, and, as with extreme weather, the coherent structures are often the features of interest.
Our approach seeks to understand the physical nature of coherent structures so that we can discover and identify them in spatiotemporal systems. It is difficult however, if not impossible, to give an actionable definition of coherent structures as the solution of a general mathematical coherence principle derived from equations of motion. Identifying and predicting complex emergent behaviors starting from fundamental laws is typically infeasible . For example, despite knowing the equations of hydrodynamics and thermodynamics, which critically govern the dynamics of hurricanes, many aspects of how hurricanes form and evolve are still poorly understood .
), which lie at the interface of physics and machine learning, provide a new means of scientific discovery directly from data. Our approach to unsupervised extreme weather event detection is through a behavior-driven theory of coherent structures in spatiotemporal systems. Below we give some basics of the theory then demonstrate its utility by identifying known coherent structures in 2D turbulence simulation data and observational data of Jupiter’s clouds from the NASA Cassini spacecraft. Finally, we show promising results on CAM5.1 water vapor data and outline the path to extreme weather event segmentation masks.
Ii Method: Local Causal States
Our behavior-driven theory of coherent structures builds on a more general theory of pattern and structure in natural systems. A quantitative theory of structure need be probabilistic, capturing structure in ensembles of behavior, and algebraic, generalizing from the group-theoretic formalism of exact symmetry to the semi-group algebra of finite-state machines. The mathematical representation of the structure of a system’s dynamical behavior is given by a minimal, optimally predictive, stochastic model [31, 10, 26]. For a model to optimally predict with minimal resources that model must capture pattern and structure present in the system’s behaviors.
Computational mechanics  makes this idea operational through the causal equivalence relation;
The equivalence classes over pasts induced by the causal equivalence relation are known as the causal states of the system; they are the unique minimal sufficient statistic of the past for optimally predicting the future.
For spatiotemporal systems, lightcones are used as local notions of past and futures. Two past lightcones and are causally equivalent if they have the same conditional distribution over future lightcones;
The resulting equivalence classes are called local causal states . They are the unique minimal sufficient statistic of past lightcones for optimal prediction of future lightcones. The -function, which generates the causal equivalence classes, maps from past lightcones to local causal states; . Segmentation is achieved by mapping a spacetime field to its associated local causal state field : every feature is mapped to its classification label (local causal state) via its past lightcone . Crucially, this ensures the global latent variable field maintains the same geometry of such that is the local latent variable corresponding to the local observable .
For real-valued systems, such as the fluid flows considered here, local causal state reconstruction requires a discretization to empirically estimate
. We use K-Means to cluster over lightcones with the lightcone distance metric
are flattened lightcone vectors,is the temporal depth of the lightcone vector at index , and is the temporal decay rate ( can be thought of as a coherence time).
Iii Results: Structural Segmentation
Because the local causal state latent variables are designed to capture structure in spatiotemporal systems, we call this level of segmentation a structural segmentation. That is, the classification assignments are labels of unique local causal states. This is an intermediate step for coherent structure segmentation (e.g. classification assignments of ‘cylcone’, ‘atmospheric river’, ‘background’, etc.), for which an additional layer of analysis is needed on top of the local causal states . From comparison with established Lagrangian Coherent Structure results we will now show that physically meaningful coherent structures are captured by our structural segmentation, and then we outline how extreme weather events may be extracted from structural segmentation of global climate data.
Building on the realization that relatively low-dimensional chaotic attractors underlies turbulent fluid flows [23, 2], the Lagrangian Coherent Structure (LCS) approach is grounded in nonlinear dynamical systems theory and seeks to describe the most repelling, attracting, and shearing material surfaces that form the skeletons of Lagrangian particle dynamics . LCS are conjectured to capture localized, emergent structures that organize the large-scale flow. We directly compare our results with the geodesic and LAVD approaches (described below) on the 2D turbulence data set from  and the Jupiter data set from  and .
There are three classes of flow structures in the LCS framework; elliptic LCS are rotating vortex-like structures, parabolic LCS are generalized Lagrangian jet-cores, and hyperbolic LCS are tendril-like stable-unstable manifolds in the flow. The geodesic approach [14, 12] is the state-of-the-art method designed to capture all three classes of LCS and has a nice interpretation for the structures it captures in terms of characteristic deformations of material surfaces. The Lagrangian-Averaged Vorticity Deviation (LAVD)  is the state-of-the-art method specifically for elliptic LCS, but is not designed to capture parabolic or hyperbolic LCS.
Iii-a 2D Turbulence
While still complex and multi-scale, the idealized 2D turbulence data provides the cleanest identification of Lagrangian Coherent Structures using our structural segmentation. Figure 1 (a) shows a snapshot of the vorticity field, and (b) and (c) show corresponding snapshots from structural segmentations using different reconstruction parameter values. To reveal finer structural details that persist on shorter time scales, Figure 1 (b) uses and . To isolate the coherent vortices, which persist at longer time scales, Figure 1 (c) was produced using and . As can be seen in (b), the local causal states distinguish between positive and negative vortices, so for (c) we modded out this symmetry by reconstructing from the absolute value of vorticity.
All three images are annotated with color-coded bounding boxes outlining elliptic LCS to directly compare with the geodesic and LAVD LCS results from Figure 9, (k) and (l) respectively, in . Green boxes are vortices identified by both the geodesic and LAVD methods and red boxes are additional vortices identified by LAVD but not the geodesic. Yellow boxes are new structural signatures of elliptic LCS discovered by the local causal states.
Because there is a single background state in (c), colored white, all states not colored white can be assigned a semantic label of coherent structure since they satisfy the local causal state definition given in  as spatially localized, temporally persistent deviations from generalized spacetime symmetries. Significantly, our method has discovered vortices in the observable field (a) as coherent structures due to the shared geometry with the latent field in (c) where they are identified as locally broken symmetries.
In the finer-scale structural segmentation of (b) we still have states outlining the coherent vortices, as we would expect. If they persist on longer time scales, they will also persist on the short time scale. The additional structure of the background potential flow largely follows the hyperbolic LCS stable-unstable manifolds. Because they act as transport barriers, they partition the flow on either side and these partitions are given by two distinct local causal states with the boundary between them running along the hyperbolic LCS in the unstable direction. For example, the narrow dark blue-colored state in the upper right of (b) indicates a narrow flow channel squeezed between two hyperbolic LCS.
Figure 1 (d) shows a snapshot from the Jupiter cloud data, with corresponding structural segmentation snapshot in (e). The Great Red Spot, highlighted with a blue arrow, is the most famous structure in Jupiter’s atmosphere. As it is a giant vortex, the Great Red Spot is identified as an elliptic LCS by both the geodesic and LAVD methods [12, 11]. While the local causal states in (e) do not capture the Great Red Spot as cleanly as the vortices in (b) and (c), it does have the same nested state structures as the turbulence vortices. There are other smaller vortices in Jupiter’s atmosphere, most notably the “string of pearls” in the Southern Temperate Belt, four of which are highlighted with blue bounding boxes. We can see in (e) that the pearls are nicely captured by the local causal states, similar to the turbulence vortices in (b).
Perhaps the most distinctive features of Jupiter’s atmosphere are the zonal belts. The east-west zonal jet streams that form the boundaries between bands are of particular relevance to Lagrangian Coherent Structure analysis. Figure 11 in  uses the geodesic method to identify these jet streams as shearless parabolic LCS, indicating they act as transport barriers that separate the zonal belts. The particular segmentation shown in (e) captures a fair amount of detail inside the bands, but the edges of the bands have neighboring pairs of local causal states with boundaries that extend contiguously in the east-west direction along the parabolic LCS transport barriers. Two such local causal state boundaries are highlighted in green, for comparison with Figure 11 (a) in . The topmost green line, in the center of (d) and (e), is the southern equatorial jet, shown in more detail in Figure 11 (b) and Figure 12 of . Its north-south meandering is clearly captured by the local causal states.
Iii-C Extreme Weather Events
The strong qualitative correspondence between local causal state structural segmentation and LCS gives validation that our method can capture meaningful structure in complex spatiotemporal systems. Our aim now is to use the structural segmentation to build extreme weather segmentation masks for climate data. Each event, e.g. hurricanes or atmospheric rivers (ARs), are identified as a unique set of structured behaviors that are captured by the local causal states in a structural segmentation.
For example, a structural segmentation of the water vapor field of the CAM5.1 global atmospheric model is shown on YouTube  and in Figure 1 (e), (f). While signatures of hurricanes and ARs are visually apparent, these events are not uniquely identifiable from the local causal states. However, hurricanes and ARs have characteristic structural signatures in other physical fields, including thermodynamic quantities such as temperature and pressure. So it is not surprising that they can not be uniquely identified from a structural segmentation of the water vapor field alone; this would be akin to describing hurricanes as just local concentrations of water vapor.
In addition to further optimization and scaling, we are currently working on implementing multi-variate local causal state reconstruction to incorporate additional physical fields. Using, for example, structural segmentation of vorticity, temperature, pressure, and water vapor fields we will be able to identify hurricanes as high rotation objects with a warm, low pressure core that locally concentrate water vapor. Similarly, the inclusion of water vapor transport will help identify ARs, as well as the use of larger lightcone templates that will better capture their large-scale geometry.
Though our structural segmentation requires information from multiple physical observables to identify extreme weather events, the generality of the local causal states will allow us to do this. An automated, objective identification of sets of local causal states across the various physical observables that uniquely corresponds to particular extreme weather events will be challenging but, we believe, achievable.
Adam Rupe and Jim Crutchfield would like to acknowledge Intel® for supporting the IPCC at UC Davis. Prabhat and Karthik Kashinath were supported by the Intel® Big Data Center. This research is based upon work supported by, or in part by, the U. S. Army Research Laboratory and the U. S. Army Research Office under contract W911NF- 13-1-0390, and used resources of the National Energy Research Scientific Computing Center, a DOE Office of Science User Facility supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231.
-  (1972) More is different. Science 177 (4047), pp. 393–396. Cited by: §I.
-  (1983) Low-dimensional chaos in a hydrodynamic system. Phys. Rev. Lett. 51, pp. 1442. Cited by: §III.
-  (2019) Spherical CNNs on unstructured grids. In International Conference on Learning Representations, Cited by: §I.
-  (2019-09–15 Jun) Gauge equivariant convolutional networks and the icosahedral CNN. In Proceedings of the 36th International Conference on Machine Learning, K. Chaudhuri and R. Salakhutdinov (Eds.), Proceedings of Machine Learning Research, Vol. 97, Long Beach, California, USA, pp. 1321–1330. Cited by: §I.
-  (2012) Between order and chaos. Nature Physics 8 (January), pp. 17–24. External Links: Cited by: §II.
-  (1987) The dependence of hurricane intensity on climate. Nature 326 (6112), pp. 483. Cited by: §I.
-  (2003) Tropical cyclones. Annual review of earth and planetary sciences 31 (1), pp. 75–104. Cited by: §I.
A big data guide to understanding climate change: the case for theory-guided data science. Big data 2 (3), pp. 155–163. External Links: Cited by: §I.
-  (2012) LICORS: light cone reconstruction of states for non-parametric forecasting of spatio-temporal systems. arXiv:1206.2398. Cited by: §II.
-  (1986) Toward a quantitative theory of self-generated complexity. Intl. J. Theo. Phys. 25, pp. 907. Cited by: §II.
-  (2017) A critical comparison of Lagrangian methods for coherent structure detection. Chaos 27 (5), pp. 053104. Cited by: §III-A, §III-B, §III.
-  (2016) Geodesic transport barriers in Jupiter’s atmosphere: video-based analysis. Siam Review 58 (1), pp. 69–89. Cited by: §III-B, §III-B, §III, §III.
-  (2016) Defining coherent vortices objectively from the vorticity. Journal of Fluid Mechanics 795, pp. 136–173. Cited by: §III.
-  (2015) Lagrangian coherent structures. Ann. Rev. Fluid Mech. 47, pp. 137–162. Cited by: §I, §III, §III.
-  (2012) Turbulence, coherent structures, dynamical systems and symmetry. Cambridge university press. Cited by: §I.
-  (2018) Exascale deep learning for climate analytics. In Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, pp. 51. Cited by: §I.
Segmenting and tracking extreme climate events using neural networks. In Deep Learning for Physical Sciences (DLPS) Workshop, held with NIPS Conference, Cited by: §I.
-  (2018) The building blocks of interpretability. Distill. Note: https://distill.pub/2018/building-blocks External Links: Cited by: §I.
ClimateNet: bringing the power of deep learning to the climate community via open datasets and architectures.
https://www.nersc.gov/research-and-development/data-analytics/big-data-center/climatenet/ Cited by: §I.
-  (2012) TECA: a parallel toolkit for extreme climate analysis. Procedia Computer Science 9, pp. 866–876. Cited by: §I.
-  (Accessed: 2019-04-10) Project disco segmentation videos. Note: https://www.youtube.com/channel/UCwKTJloOOFQHVHDwkpqIdYA Cited by: Fig. 1, §III-C.
-  (2018) Entropy-based generating Markov partitions for complex systems. Chaos: An Interdisciplinary Journal of Nonlinear Science 28 (3), pp. 033611. External Links: Cited by: §I.
-  (1971) On the nature of turbulence. Comm. Math. Phys. 20, pp. 167–192. Cited by: §III.
-  (2015) Identifying causal gateways and mediators in complex spatio-temporal systems. Nature communications 6, pp. 8502. Cited by: §I.
-  (2018) Local causal states and discrete coherent structures. Chaos 28 (7), pp. 1–22. External Links: Cited by: §III-A, §III.
-  (2001) Computational mechanics: pattern and prediction, structure and simplicity. J. Stat. Phys. 104, pp. 817–879. Cited by: §II.
-  (2003) Optimal nonlinear prediction of random fields on networks. Discrete Mathematics & Theoretical Computer Science. Cited by: §II.
-  (2018) Atmospheric river tracking method intercomparison project (ARTMIP): project goals and experimental design. Geoscientific Model Development 11 (6), pp. 2455–2474. External Links: Cited by: §I.
-  (2005) Changes in tropical cyclone number, duration, and intensity in a warming environment. Science 309 (5742), pp. 1844–1846. Cited by: §I.
-  (2015) A data-driven approximation of the Koopman operator: extending dynamic mode decomposition. Journal of Nonlinear Science 25 (6), pp. 1307–1346. Cited by: §I.
-  (1984) Computation theory of cellular automata. Comm. Math. Phys. 96, pp. 15. Cited by: §II.
-  (2019) Causal deconvolution by algorithmic generative models. Nature Machine Intelligence 1 (1), pp. 58. Cited by: §I.