Do language models learn typicality judgments from text?

05/06/2021
by   Kanishka Misra, et al.
0

Building on research arguing for the possibility of conceptual and categorical knowledge acquisition through statistics contained in language, we evaluate predictive language models (LMs) – informed solely by textual input – on a prevalent phenomenon in cognitive science: typicality. Inspired by experiments that involve language processing and show robust typicality effects in humans, we propose two tests for LMs. Our first test targets whether typicality modulates LM probabilities in assigning taxonomic category memberships to items. The second test investigates sensitivities to typicality in LMs' probabilities when extending new information about items to their categories. Both tests show modest – but not completely absent – correspondence between LMs and humans, suggesting that text-based exposure alone is insufficient to acquire typicality knowledge.

READ FULL TEXT

page 4

page 5

research
05/13/2022

A Property Induction Framework for Neural Language Models

To what extent can experience from language contribute to our conceptual...
research
11/04/2021

On Semantic Cognition, Inductive Generalization, and Language Models

My doctoral research focuses on understanding semantic knowledge in neur...
research
04/22/2021

Finding Fuzziness in Neural Network Models of Language Processing

Humans often communicate by using imprecise language, suggesting that fu...
research
07/31/2023

LLMs4OL: Large Language Models for Ontology Learning

We propose the LLMs4OL approach, which utilizes Large Language Models (L...
research
08/06/2023

Why Linguistics Will Thrive in the 21st Century: A Reply to Piantadosi (2023)

We present a critical assessment of Piantadosi's (2023) claim that "Mode...
research
05/12/2022

Predicting Human Psychometric Properties Using Computational Language Models

Transformer-based language models (LMs) continue to achieve state-of-the...
research
02/23/2019

Categorization in the Wild: Generalizing Cognitive Models to Naturalistic Data across Languages

Categories such as animal or furniture are acquired at an early age and ...

Please sign up or login with your details

Forgot password? Click here to reset