Large Linguistic Models: Analyzing theoretical linguistic abilities of LLMs

05/01/2023
by   Gašper Beguš, et al.
45

The performance of large language models (LLMs) has recently improved to the point where the models can generate valid and coherent meta-linguistic analyses of data. This paper illustrates a vast potential for analyses of the meta-linguistic abilities of large language models. LLMs are primarily trained on language data in the form of text; analyzing their meta-linguistic abilities is informative both for our understanding of the general capabilities of LLMs as well as for models of linguistics. In this paper, we propose several types of experiments and prompt designs that allow us to analyze the ability of GPT-4 to generate meta-linguistic analyses. We focus on three linguistics subfields with formalisms that allow for a detailed analysis of GPT-4's theoretical capabilities: theoretical syntax, phonology, and semantics. We identify types of experiments, provide general guidelines, discuss limitations, and offer future directions for this research program.

READ FULL TEXT

page 6

page 7

page 9

page 10

page 12

page 15

page 16

page 17

research
06/12/2023

Large language models and (non-)linguistic recursion

Recursion is one of the hallmarks of human language. While many design f...
research
08/18/2020

Password Guessers Under a Microscope: An In-Depth Analysis to Inform Deployments

Password guessers are instrumental for assessing the strength of passwor...
research
04/28/2023

Are Emergent Abilities of Large Language Models a Mirage?

Recent work claims that large language models display emergent abilities...
research
09/12/2023

BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Models

The rapid development of Large Language Models (LLMs) and the emergence ...
research
06/11/2023

A blind spot for large language models: Supradiegetic linguistic information

Large Language Models (LLMs) like ChatGPT reflect profound changes in th...
research
05/16/2022

What GPT Knows About Who is Who

Coreference resolution – which is a crucial task for understanding disco...
research
02/19/2022

Is there an aesthetic component of language?

Speakers of all human languages make use of grammatical devices to expre...

Please sign up or login with your details

Forgot password? Click here to reset