Sensitivity Measures Based on Scoring Functions

03/01/2022
by   Tobias Fissler, et al.
0

We propose a holistic framework for constructing sensitivity measures for any elicitable functional T of a response variable. The sensitivity measures, termed score-based sensitivities, are constructed via scoring functions that are (strictly) consistent for T. These score-based sensitivities quantify the relative improvement in predictive accuracy when available information, e.g., from explanatory variables, is used ideally. We establish intuitive and desirable properties of these sensitivities and discuss advantageous choices of scoring functions leading to scale-invariant sensitivities. Since elicitable functionals typically possess rich classes of (strictly) consistent scoring functions, we demonstrate how Murphy diagrams can provide a picture of all score-based sensitivity measures. We discuss the family of score-based sensitivities for the mean functional (of which the Sobol indices are a special case) and risk functionals such as Value-at-Risk, and the pair Value-at-Risk and Expected Shortfall. The sensitivity measures are illustrated using numerous examples, including the Ishigami–Homma test function. In a simulation study, estimation of score-based sensitivities for a non-linear insurance portfolio is performed using neural nets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/12/2019

Elicitability of Range Value at Risk

The predictive performance of point forecasts for a statistical function...
research
11/27/2017

Order-Sensitivity and Equivariance of Scoring Functions

The relative performance of competing point forecasts is usually measure...
research
07/28/2022

Model selection with Gini indices under auto-calibration

The Gini index does not give a strictly consistent scoring rule in gener...
research
07/27/2017

A Family of Metrics for Clustering Algorithms

We give the motivation for scoring clustering algorithms and a metric M ...
research
07/02/2019

Elicitability and Identifiability of Systemic Risk Measures and other Set-Valued Functionals

This paper is concerned with a two-fold objective. Firstly, we establish...
research
08/29/2021

A scoring framework for tiered warnings and multicategorical forecasts based on fixed risk measures

The use of tiered warnings and multicategorical forecasts are ubiquitous...
research
06/29/2021

Isotonic regression for functionals of elicitation complexity greater than one

We study the non-parametric isotonic regression problem for bivariate el...

Please sign up or login with your details

Forgot password? Click here to reset