On the Existence of Tacit Assumptions in Contextualized Language Models

04/10/2020
by   Nathaniel Weir, et al.
0

Humans carry stereotypic tacit assumptions (STAs) (Prince, 1978), or propositional beliefs about generic concepts. Such associations are crucial for understanding natural language. We construct a diagnostic set of word prediction prompts to evaluate whether recent neural contextualized language models trained on large text corpora capture STAs. Our prompts are based on human responses in a psychological study of conceptual associations. We find models to be profoundly effective at retrieving concepts given associated properties. Our results demonstrate empirical evidence that stereotypic conceptual representations are captured in neural models derived from semi-supervised linguistic exposure.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/23/2022

Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Models

NLP models trained on text have been shown to reproduce human stereotype...
research
07/18/2023

Large Language Models Perform Diagnostic Reasoning

We explore the extension of chain-of-thought (CoT) prompting to medical ...
research
06/30/2023

Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblocks

We consider the questions of whether or not large language models (LLMs)...
research
10/02/2014

Not All Neural Embeddings are Born Equal

Neural language models learn word representations that capture rich ling...
research
12/05/2021

Interpretable Privacy Preservation of Text Representations Using Vector Steganography

Contextual word representations generated by language models (LMs) learn...
research
05/12/2022

Predicting Human Psychometric Properties Using Computational Language Models

Transformer-based language models (LMs) continue to achieve state-of-the...
research
02/08/2022

Semantic features of object concepts generated with GPT-3

Semantic features have been playing a central role in investigating the ...

Please sign up or login with your details

Forgot password? Click here to reset