Validation of point process predictions with proper scoring rules

We introduce a class of proper scoring rules for evaluating spatial point process forecasts based on summary statistics. These scoring rules rely on Monte-Carlo approximations of expectations and can therefore easily be evaluated for any point process model that can be simulated. In this regard, they are more flexible than the commonly used logarithmic score and other existing proper scores for point process predictions. The scoring rules allow for evaluating the calibration of a model to specific aspects of a point process, such as its spatial distribution or tendency towards clustering. Using simulations we analyze the sensitivity of our scoring rules to different aspects of the forecasts and compare it to the logarithmic score. Applications to earthquake occurrences in northern California, USA and the spatial distribution of Pacific silver firs in Findley Lake Reserve in Washington, USA highlight the usefulness of our scores for scientific model selection.

READ FULL TEXT
research
12/23/2020

Beyond Strictly Proper Scoring Rules: The Importance of Being Local

The evaluation of probabilistic forecasts plays a central role both in t...
research
01/29/2021

Evaluating the Discrimination Ability of Proper Multivariate Scoring Rules

Proper scoring rules are commonly applied to quantify the accuracy of di...
research
08/22/2018

Calibration Scoring Rules for Practical Prediction Training

In situations where forecasters are scored on the quality of their proba...
research
07/27/2017

A Family of Metrics for Clustering Algorithms

We give the motivation for scoring clustering algorithms and a metric M ...
research
08/18/2023

Probabilistic Rainfall Downscaling: Joint Generalized Neural Models with Censored Spatial Gaussian Copula

This work introduces a novel approach for generating conditional probabi...
research
06/10/2018

Forecast evaluation with imperfect observations and imperfect models

The field of statistics has become one of the mathematical foundations i...
research
11/11/2021

Proper Scoring Rules, Gradients, Divergences, and Entropies for Paths and Time Series

Many forecasts consist not of point predictions but concern the evolutio...

Please sign up or login with your details

Forgot password? Click here to reset