A fully data-driven method to identify (correlated) changes in diachronic corpora

08/26/2015
by   Alexander Koplenig, et al.
0

In this paper, a method for measuring synchronic corpus (dis-)similarity put forward by Kilgarriff (2001) is adapted and extended to identify trends and correlated changes in diachronic text data, using the Corpus of Historical American English (Davies 2010a) and the Google Ngram Corpora (Michel et al. 2010a). This paper shows that this fully data-driven method, which extracts word types that have undergone the most pronounced change in frequency in a given period of time, is computationally very cheap and that it allows interpretations of diachronic trends that are both intuitively plausible and motivated from the perspective of information theory. Furthermore, it demonstrates that the method is able to identify correlated linguistic changes and diachronic shifts that can be linked to historical events. Finally, it can help to improve diachronic POS tagging and complement existing NLP approaches. This indicates that the approach can facilitate an improved understanding of diachronic processes in language change.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 17

page 28

02/24/2020

Parsing Early Modern English for Linguistic Search

We investigate the question of whether advances in NLP over the last few...
08/13/2021

Diachronic Analysis of German Parliamentary Proceedings: Ideological Shifts through the Lens of Political Biases

We analyze bias in historical corpora as encoded in diachronic distribut...
06/02/2018

Quantifying the dynamics of topical fluctuations in language

The availability of large diachronic corpora has provided the impetus fo...
11/03/2018

Challenges in detecting evolutionary forces in language change using diachronic corpora

Newberry et al. (Detecting evolutionary forces in language change, Natur...
09/30/2016

Modeling Language Change in Historical Corpora: The Case of Portuguese

This paper presents a number of experiments to model changes in a histor...
11/06/2019

Gextext: Disease Network Extraction from Biomedical Literature

PURPOSE: We propose a fully unsupervised method to learn latent disease ...
06/04/2018

History Playground: A Tool for Discovering Temporal Trends in Massive Textual Corpora

Recent studies have shown that macroscopic patterns of continuity and ch...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.