DeepAI AI Chat
Log In Sign Up

Spatial machine-learning model diagnostics: a model-agnostic distance-based approach

11/13/2021
by   Alexander Brenning, et al.
0

While significant progress has been made towards explaining black-box machine-learning (ML) models, there is still a distinct lack of diagnostic tools that elucidate the spatial behaviour of ML models in terms of predictive skill and variable importance. This contribution proposes spatial prediction error profiles (SPEPs) and spatial variable importance profiles (SVIPs) as novel model-agnostic assessment and interpretation tools for spatial prediction models with a focus on prediction distance. Their suitability is demonstrated in two case studies representing a regionalization task in an environmental-science context, and a classification task from remotely-sensed land cover classification. In these case studies, the SPEPs and SVIPs of geostatistical methods, linear models, random forest, and hybrid algorithms show striking differences but also relevant similarities. Limitations of related cross-validation techniques are outlined, and the case is made that modelers should focus their model assessment and interpretation on the intended spatial prediction horizon. The range of autocorrelation, in contrast, is not a suitable criterion for defining spatial cross-validation test sets. The novel diagnostic tools enrich the toolkit of spatial data science, and may improve ML model interpretation, selection, and design.

READ FULL TEXT

page 1

page 2

page 3

page 4

07/25/2022

Machine Learning to Predict the Antimicrobial Activity of Cold Atmospheric Plasma-Activated Liquids

Plasma is defined as the fourth state of matter and non-thermal plasma c...
05/16/2020

Predicting into unknown space? Estimating the area of applicability of spatial prediction models

Predictive modelling using machine learning has become very popular for ...
06/02/2020

Local Interpretability of Calibrated Prediction Models: A Case of Type 2 Diabetes Mellitus Screening Test

Machine Learning (ML) models are often complex and difficult to interpre...
06/09/2018

A hybrid econometric-machine learning approach for relative importance analysis: Food inflation

A measure of relative importance of variables is often desired by resear...