DeepAI AI Chat
Log In Sign Up

Scoring rules in survival analysis

by   Raphael Sonabend, et al.

Scoring rules promote rational and good decision making and predictions by models, this is increasingly important for automated procedures of `auto-ML'. The Brier score and Log loss are well-established scoring rules for classification and regression and possess the `strict properness' property that encourages optimal predictions. In this paper we survey proposed scoring rules for survival analysis, establish the first clear definition of `(strict) properness' for survival scoring rules, and determine which losses are proper and improper. We prove that commonly utilised scoring rules that are claimed to be proper are in fact improper. We further prove that under a strict set of assumptions a class of scoring rules is strictly proper for, what we term, `approximate' survival losses. We hope these findings encourage further research into robust validation of survival models and promote honest evaluation.


page 1

page 2

page 3

page 4


Scoring Rules for Performative Binary Prediction

We construct a model of expert prediction where predictions can influenc...

Multidimensional Service Quality Scoring System

This supplementary paper aims to introduce the Multidimensional Service ...

Validation of point process predictions with proper scoring rules

We introduce a class of proper scoring rules for evaluating spatial poin...

On loss functions and regret bounds for multi-category classification

We develop new approaches in multi-class settings for constructing prope...

Calibration Scoring Rules for Practical Prediction Training

In situations where forecasters are scored on the quality of their proba...

Binary Scoring Rules that Incentivize Precision

All proper scoring rules incentivize an expert to predict accurately (re...

From Classification Accuracy to Proper Scoring Rules: Elicitability of Probabilistic Top List Predictions

In the face of uncertainty, the need for probabilistic assessments has l...