A fully data-driven method to identify (correlated) changes in diachronic corpora

by   Alexander Koplenig, et al.

In this paper, a method for measuring synchronic corpus (dis-)similarity put forward by Kilgarriff (2001) is adapted and extended to identify trends and correlated changes in diachronic text data, using the Corpus of Historical American English (Davies 2010a) and the Google Ngram Corpora (Michel et al. 2010a). This paper shows that this fully data-driven method, which extracts word types that have undergone the most pronounced change in frequency in a given period of time, is computationally very cheap and that it allows interpretations of diachronic trends that are both intuitively plausible and motivated from the perspective of information theory. Furthermore, it demonstrates that the method is able to identify correlated linguistic changes and diachronic shifts that can be linked to historical events. Finally, it can help to improve diachronic POS tagging and complement existing NLP approaches. This indicates that the approach can facilitate an improved understanding of diachronic processes in language change.


page 17

page 28


Parsing Early Modern English for Linguistic Search

We investigate the question of whether advances in NLP over the last few...

Diachronic Analysis of German Parliamentary Proceedings: Ideological Shifts through the Lens of Political Biases

We analyze bias in historical corpora as encoded in diachronic distribut...

Letters From the Past: Modeling Historical Sound Change Through Diachronic Character Embeddings

While a great deal of work has been done on NLP approaches to lexical se...

Reliable identification of selection mechanisms in language change

Language change is a cultural evolutionary process in which variants of ...

Quantifying the dynamics of topical fluctuations in language

The availability of large diachronic corpora has provided the impetus fo...

Challenges in detecting evolutionary forces in language change using diachronic corpora

Newberry et al. (Detecting evolutionary forces in language change, Natur...

Modeling Language Change in Historical Corpora: The Case of Portuguese

This paper presents a number of experiments to model changes in a histor...

Please sign up or login with your details

Forgot password? Click here to reset