Language Models Use Monotonicity to Assess NPI Licensing

05/28/2021
by   Jaap Jumelet, et al.
0

We investigate the semantic knowledge of language models (LMs), focusing on (1) whether these LMs create categories of linguistic environments based on their semantic monotonicity properties, and (2) whether these categories play a similar role in LMs as in human language understanding, using negative polarity item licensing as a case study. We introduce a series of experiments consisting of probing with diagnostic classifiers (DCs), linguistic acceptability tasks, as well as a novel DC ranking method that tightly connects the probing results to the inner workings of the LM. By applying our experimental pipeline to LMs trained on various filtered corpora, we are able to gain stronger insights into the semantic generalizations that are acquired by these models.

READ FULL TEXT
research
09/08/2021

Transformers in the loop: Polarity in neural models of language

Representation of linguistic phenomena in computational language models ...
research
06/21/2023

Feature Interactions Reveal Linguistic Structure in Language Models

We study feature interactions in the context of feature attribution meth...
research
07/01/2022

Is neural language acquisition similar to natural? A chronological probing study

The probing methodology allows one to obtain a partial representation of...
research
06/21/2023

Limits for Learning with Language Models

With the advent of large language models (LLMs), the trend in NLP has be...
research
12/13/2022

A fine-grained comparison of pragmatic language understanding in humans and language models

Pragmatics is an essential part of communication, but it remains unclear...
research
06/12/2021

Can Transformer Language Models Predict Psychometric Properties?

Transformer-based language models (LMs) continue to advance state-of-the...
research
05/15/2023

Using LLM-assisted Annotation for Corpus Linguistics: A Case Study of Local Grammar Analysis

Chatbots based on Large Language Models (LLMs) have shown strong capabil...

Please sign up or login with your details

Forgot password? Click here to reset