When Bert Forgets How To POS: Amnesic Probing of Linguistic Properties and MLM Predictions

06/01/2020
by   Yanai Elazar, et al.
0

A growing body of work makes use of probing in order to investigate the working of neural models, often considered black boxes. Recently, an ongoing debate emerged surrounding the limitations of the probing paradigm. In this work, we point out the inability to infer behavioral conclusions from probing results, and offer an alternative method which is focused on how the information is being used, rather than on what information is encoded. Our method, Amnesic Probing, follows the intuition that the utility of a property for a given task can be assessed by measuring the influence of a causal intervention which removes it from the representation. Equipped with this new analysis tool, we can now ask questions that were not possible before, e.g. is part-of-speech information important for word prediction? We perform a series of analyses on BERT to answer these types of questions. Our findings demonstrate that conventional probing performance is not correlated to task importance, and we call for increased scrutiny of claims that draw behavioral or causal conclusions from probing results.

READ FULL TEXT
research
05/14/2021

Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Prediction

When language models process syntactically complex sentences, do they us...
research
05/14/2022

Naturalistic Causal Probing for Morpho-Syntax

Probing has become a go-to methodology for interpreting and analyzing de...
research
04/19/2022

Probing for the Usage of Grammatical Number

A central quest of probing is to uncover how pre-trained models encode a...
research
11/06/2019

Learning to Answer by Learning to Ask: Getting the Best of GPT-2 and BERT Worlds

Automatic question generation aims at the generation of questions from a...
research
03/15/2012

Causal Conclusions that Flip Repeatedly and Their Justification

Over the past two decades, several consistent procedures have been desig...
research
06/05/2019

Neural Legal Judgment Prediction in English

Legal judgment prediction is the task of automatically predicting the ou...
research
06/30/2021

The MultiBERTs: BERT Reproductions for Robustness Analysis

Experiments with pretrained models such as BERT are often based on a sin...

Please sign up or login with your details

Forgot password? Click here to reset