DeepAI AI Chat
Log In Sign Up

Large Linguistic Models: Analyzing theoretical linguistic abilities of LLMs

by   Gašper Beguš, et al.

The performance of large language models (LLMs) has recently improved to the point where the models can generate valid and coherent meta-linguistic analyses of data. This paper illustrates a vast potential for analyses of the meta-linguistic abilities of large language models. LLMs are primarily trained on language data in the form of text; analyzing their meta-linguistic abilities is informative both for our understanding of the general capabilities of LLMs as well as for models of linguistics. In this paper, we propose several types of experiments and prompt designs that allow us to analyze the ability of GPT-4 to generate meta-linguistic analyses. We focus on three linguistics subfields with formalisms that allow for a detailed analysis of GPT-4's theoretical capabilities: theoretical syntax, phonology, and semantics. We identify types of experiments, provide general guidelines, discuss limitations, and offer future directions for this research program.


page 6

page 7

page 9

page 10

page 12

page 15

page 16

page 17


Beyond the limitations of any imaginable mechanism: large language models and psycholinguistics

Large language models are not detailed models of human linguistic proces...

Are Emergent Abilities of Large Language Models a Mirage?

Recent work claims that large language models display emergent abilities...

Password Guessers Under a Microscope: An In-Depth Analysis to Inform Deployments

Password guessers are instrumental for assessing the strength of passwor...

Testing AI performance on less frequent aspects of language reveals insensitivity to underlying meaning

Advances in computational methods and big data availability have recentl...

What GPT Knows About Who is Who

Coreference resolution – which is a crucial task for understanding disco...

Is there an aesthetic component of language?

Speakers of all human languages make use of grammatical devices to expre...

Fundamentals of Generative Large Language Models and Perspectives in Cyber-Defense

Generative Language Models gained significant attention in late 2022 / e...