DeepAI AI Chat
Log In Sign Up

Data Generating Process to Evaluate Causal Discovery Techniques for Time Series Data

by   Andrew R. Lawrence, et al.

Going beyond correlations, the understanding and identification of causal relationships in observational time series, an important subfield of Causal Discovery, poses a major challenge. The lack of access to a well-defined ground truth for real-world data creates the need to rely on synthetic data for the evaluation of these methods. Existing benchmarks are limited in their scope, as they either are restricted to a "static" selection of data sets, or do not allow for a granular assessment of the methods' performance when commonly made assumptions are violated. We propose a flexible and simple to use framework for generating time series data, which is aimed at developing, evaluating, and benchmarking time series causal discovery methods. In particular, the framework can be used to fine tune novel methods on vast amounts of data, without "overfitting" them to a benchmark, but rather so they perform well in real-world use cases. Using our framework, we evaluate prominent time series causal discovery methods and demonstrate a notable degradation in performance when their assumptions are invalidated and their sensitivity to choice of hyperparameters. Finally, we propose future research directions and how our framework can support both researchers and practitioners.


page 1

page 2

page 3

page 4


Neural Additive Vector Autoregression Models for Causal Discovery in Time Series Data

Causal structure discovery in complex dynamical systems is an important ...

Evaluating Temporal Observation-Based Causal Discovery Techniques Applied to Road Driver Behaviour

Autonomous robots are required to reason about the behaviour of dynamic ...

Beyond Predictions in Neural ODEs: Identification and Interventions

Spurred by tremendous success in pattern matching and prediction tasks, ...

Statistical evaluation of a long-memory process using the generalized entropic Value-at-Risk

The modeling and identification of time series data with a long memory a...

A Demonstration of Benchmarking Time Series Management Systems in the Cloud

Time Series Management Systems (TSMS) are Database Management Systems th...

Capturing Structure Implicitly from Time-Series having Limited Data

Scientific fields such as insider-threat detection and highway-safety pl...

A Ladder of Causal Distances

Causal discovery, the task of automatically constructing a causal model ...