Neural Network Attribution Methods for Problems in Geoscience: A Novel Synthetic Benchmark Dataset

by   Antonios Mamalakis, et al.

Despite the increasingly successful application of neural networks to many problems in the geosciences, their complex and nonlinear structure makes the interpretation of their predictions difficult, which limits model trust and does not allow scientists to gain physical insights about the problem at hand. Many different methods have been introduced in the emerging field of eXplainable Artificial Intelligence (XAI), which aim at attributing the network's prediction to specific features in the input domain. XAI methods are usually assessed by using benchmark datasets (like MNIST or ImageNet for image classification), or through deletion/insertion techniques. In either case, however, an objective, theoretically-derived ground truth for the attribution is lacking, making the assessment of XAI in many cases subjective. Also, benchmark datasets for problems in geosciences are rare. Here, we provide a framework, based on the use of additively separable functions, to generate attribution benchmark datasets for regression problems for which the ground truth of the attribution is known a priori. We generate a long benchmark dataset and train a fully-connected network to learn the underlying function that was used for simulation. We then compare estimated attribution heatmaps from different XAI methods to the ground truth in order to identify examples where specific XAI methods perform well or poorly. We believe that attribution benchmarks as the ones introduced herein are of great importance for further application of neural networks in the geosciences, and for accurate implementation of XAI methods, which will increase model trust and assist in discovering new science.



There are no comments yet.


page 3

page 4

page 6

page 7


Towards Rigorous Interpretations: a Formalisation of Feature Attribution

Feature attribution is often loosely presented as the process of selecti...

Explaining COVID-19 and Thoracic Pathology Model Predictions by Identifying Informative Input Features

Neural networks have demonstrated remarkable performance in classificati...

Evaluating the Correctness of Explainable AI Algorithms for Classification

Explainable AI has attracted much research attention in recent years wit...

Software for Dataset-wide XAI: From Local Explanations to Global Insights with Zennit, CoRelAy, and ViRelAy

Deep Neural Networks (DNNs) are known to be strong predictors, but their...

"Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classification

Feature attribution a.k.a. input salience methods which assign an import...

Continuous Optimization Benchmarks by Simulation

Benchmark experiments are required to test, compare, tune, and understan...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.