Inaccessible Neural Language Models Could Reinvigorate Linguistic Nativism

01/12/2023
by   Patrick Perrine, et al.
0

Large Language Models (LLMs) have been making big waves in the machine learning community within the past few years. The impressive scalability of LLMs due to the advent of deep learning can be seen as a continuation of empiricist lingusitic methods, as opposed to rule-based linguistic methods that are grounded in a nativist perspective. Current LLMs are generally inaccessible to resource-constrained researchers, due to a variety of factors including closed source code. This work argues that this lack of accessibility could instill a nativist bias in researchers new to computational linguistics, given that new researchers may only have rule-based, nativist approaches to study to produce new work. Also, given that there are numerous critics of deep learning claiming that LLMs and related methods may soon lose their relevancy, we speculate that such an event could trigger a new wave of nativism in the language processing community. To prevent such a dramatic shift and placing favor in hybrid methods of rules and deep learning, we call upon researchers to open source their LLM code wherever possible to allow both empircist and hybrid approaches to remain accessible.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/02/2023

Towards Code Watermarking with Dual-Channel Transformations

The expansion of the open source community and the rise of large languag...
research
08/25/2023

Construction Grammar and Language Models

Recent progress in deep learning and natural language processing has giv...
research
03/15/2022

Non-neural Models Matter: A Re-evaluation of Neural Referring Expression Generation Systems

In recent years, neural models have often outperformed rule-based and cl...
research
08/25/2022

Deep Learning-based approaches for automatic detection of shell nouns and evaluation on WikiText-2

In some areas, such as Cognitive Linguistics, researchers are still usin...
research
08/11/2023

Enhancing Phenotype Recognition in Clinical Notes Using Large Language Models: PhenoBCBERT and PhenoGPT

We hypothesize that large language models (LLMs) based on the transforme...
research
10/19/2020

Interpretable Machine Learning – A Brief History, State-of-the-Art and Challenges

We present a brief history of the field of interpretable machine learnin...

Please sign up or login with your details

Forgot password? Click here to reset