Bad and good errors: value-weighted skill scores in deep ensemble learning

03/04/2021
by   Sabrina Guastavino, et al.
0

In this paper we propose a novel approach to realize forecast verification. Specifically, we introduce a strategy for assessing the severity of forecast errors based on the evidence that, on the one hand, a false alarm just anticipating an occurring event is better than one in the middle of consecutive non-occurring events, and that, on the other hand, a miss of an isolated event has a worse impact than a miss of a single event, which is part of several consecutive occurrences. Relying on this idea, we introduce a novel definition of confusion matrix and skill scores giving greater importance to the value of the prediction rather than to its quality. Then, we introduce a deep ensemble learning procedure for binary classification, in which the probabilistic outcomes of a neural network are clustered via optimization of these value-weighted skill scores. We finally show the performances of this approach in the case of three applications concerned with pollution, space weather and stock prize forecasting.

READ FULL TEXT
research
09/20/2021

Prediction of severe thunderstorm events with ensemble deep learning and radar data

The problem of nowcasting extreme weather events can be addressed by app...
research
09/10/2022

On the Evaluation of Skill in Binary Forecast

A good prediction is very important for scientific, economic, and admini...
research
11/17/2019

Weather event severity prediction using buoy data and machine learning

In this paper, we predict severity of extreme weather events (tropical s...
research
09/11/2022

Weighted verification tools to evaluate univariate and multivariate forecasts for high-impact weather events

To mitigate the impacts associated with adverse weather conditions, mete...
research
02/13/2020

Ensemble neural network forecasts with singular value decomposition

Ensemble weather forecasts enable a measure of uncertainty to be attache...
research
06/05/2018

A penalty criterion for score forecasting in soccer

This note proposes a penalty criterion for assessing correct score forec...
research
07/18/2019

Artificial skill in monsoon onset prediction: two recent examples

For two cases of empirical monsoon onset prediction it is argued that cu...

Please sign up or login with your details

Forgot password? Click here to reset