Log In Sign Up

Character Distributions of Classical Chinese Literary Texts: Zipf's Law, Genres, and Epochs

by   Chao-Lin Liu, et al.

We collect 14 representative corpora for major periods in Chinese history in this study. These corpora include poetic works produced in several dynasties, novels of the Ming and Qing dynasties, and essays and news reports written in modern Chinese. The time span of these corpora ranges between 1046 BCE and 2007 CE. We analyze their character and word distributions from the viewpoint of the Zipf's law, and look for factors that affect the deviations and similarities between their Zipfian curves. Genres and epochs demonstrated their influences in our analyses. Specifically, the character distributions for poetic works of between 618 CE and 1644 CE exhibit striking similarity. In addition, although texts of the same dynasty may tend to use the same set of characters, their character distributions still deviate from each other.


page 1

page 2

page 3

page 4


Flexible Computing Services for Comparisons and Analyses of Classical Chinese Poetry

We collect nine corpora of representative Chinese poetry for the time sp...

Multiple Character Embeddings for Chinese Word Segmentation

Chinese word segmentation (CWS) is often regarded as a character-based s...

The 'Letter' Distribution in the Chinese Language

Corpus-based statistical analysis plays a significant role in linguistic...

Evolution of Part-of-Speech in Classical Chinese

Classical Chinese is a language notable for its word class flexibility: ...

SHUOWEN-JIEZI: Linguistically Informed Tokenizers For Chinese Language Model Pretraining

Conventional tokenization methods for Chinese pretrained language models...

Character Entropy in Modern and Historical Texts: Comparison Metrics for an Undeciphered Manuscript

This paper outlines the creation of three corpora for multilingual compa...