Why do you think that? Exploring Faithful Sentence-Level Rationales Without Supervision

10/07/2020
by   Max Glockner, et al.
0

Evaluating the trustworthiness of a model's prediction is essential for differentiating between `right for the right reasons' and `right for the wrong reasons'. Identifying textual spans that determine the target label, known as faithful rationales, usually relies on pipeline approaches or reinforcement learning. However, such methods either require supervision and thus costly annotation of the rationales or employ non-differentiable models. We propose a differentiable training-framework to create models which output faithful rationales on a sentence level, by solely applying supervision on the target task. To achieve this, our model solves the task based on each rationale individually and learns to assign high scores to those which solved the task best. Our evaluation on three different datasets shows competitive results compared to a standard BERT blackbox while exceeding a pipeline counterpart's performance in two cases. We further exploit the transparent decision-making process of these models to prefer selecting the correct rationales by applying direct supervision, thereby boosting the performance on the rationale-level.

READ FULL TEXT

page 1

page 2

page 3

page 4

03/10/2017

Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations

Neural networks are among the most accurate supervised learning methods ...
10/14/2021

Making Document-Level Information Extraction Right for the Right Reasons

Document-level information extraction is a flexible framework compatible...
01/15/2020

Right for the Wrong Scientific Reasons: Revising Deep Networks by Interacting with their Explanations

Deep neural networks have shown excellent performances in many real-worl...
11/17/2017

Automatic Pill Reminder for Easy Supervision

In this paper we present a working model of an automatic pill reminder a...
10/13/2021

Automatic DJ Transitions with Differentiable Audio Effects and Generative Adversarial Networks

A central task of a Disc Jockey (DJ) is to create a mixset of mu-sic wit...
08/22/2020

Supervision Levels Scale (SLS)

We propose a three-dimensional discrete and incremental scale to encode ...