Character Distributions of Classical Chinese Literary Texts: Zipf's Law, Genres, and Epochs

09/17/2017
by   Chao-Lin Liu, et al.
0

We collect 14 representative corpora for major periods in Chinese history in this study. These corpora include poetic works produced in several dynasties, novels of the Ming and Qing dynasties, and essays and news reports written in modern Chinese. The time span of these corpora ranges between 1046 BCE and 2007 CE. We analyze their character and word distributions from the viewpoint of the Zipf's law, and look for factors that affect the deviations and similarities between their Zipfian curves. Genres and epochs demonstrated their influences in our analyses. Specifically, the character distributions for poetic works of between 618 CE and 1644 CE exhibit striking similarity. In addition, although texts of the same dynasty may tend to use the same set of characters, their character distributions still deviate from each other.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/18/2017

Flexible Computing Services for Comparisons and Analyses of Classical Chinese Poetry

We collect nine corpora of representative Chinese poetry for the time sp...
research
08/15/2018

Multiple Character Embeddings for Chinese Word Segmentation

Chinese word segmentation (CWS) is often regarded as a character-based s...
research
05/26/2020

The 'Letter' Distribution in the Chinese Language

Corpus-based statistical analysis plays a significant role in linguistic...
research
09/23/2020

Evolution of Part-of-Speech in Classical Chinese

Classical Chinese is a language notable for its word class flexibility: ...
research
06/01/2021

SHUOWEN-JIEZI: Linguistically Informed Tokenizers For Chinese Language Model Pretraining

Conventional tokenization methods for Chinese pretrained language models...
research
10/28/2020

Character Entropy in Modern and Historical Texts: Comparison Metrics for an Undeciphered Manuscript

This paper outlines the creation of three corpora for multilingual compa...
research
07/22/2020

When Classical Chinese Meets Machine Learning: Explaining the Relative Performances of Word and Sentence Segmentation Tasks

We consider three major text sources about the Tang Dynasty of China in ...

Please sign up or login with your details

Forgot password? Click here to reset