Classical Chinese Sentence Segmentation for Tomb Biographies of Tang Dynasty

by   Chao-Lin Liu, et al.

Tomb biographies of the Tang dynasty provide invaluable information about Chinese history. The original biographies are classical Chinese texts which contain neither word boundaries nor sentence boundaries. Relying on three published books of tomb biographies of the Tang dynasty, we investigated the effectiveness of employing machine-learning methods for algorithmically identifying the pauses and terminals of sentences in the biographies. We consider the segmentation task as a classification problem. Chinese characters that are and are not followed by a punctuation mark are classified into two categories. We applied a machine-learning-based mechanism, the conditional random fields (CRF), to classify the characters (and words) in the texts, and we studied the contributions of selected types of lexical information to the resulting quality of the segmentation recommendations. This proposal presented at the DH 2018 conference discussed some of the basic experiments and their evaluations. By considering the contextual information and employing the heuristics provided by experts of Chinese literature, we achieved F1 measures that were better than 80 employ deep neural networks helped us further improve the results in recent work.



There are no comments yet.


page 1


Sentence Segmentation for Classical Chinese Based on LSTM with Radical Embedding

In this paper, we develop a low than character feature embedding called ...

Generating Adversarial Examples in Chinese Texts Using Sentence-Pieces

Adversarial attacks in texts are mostly substitution-based methods that ...

Word Segmentation as Graph Partition

We propose a new approach to the Chinese word segmentation problem that ...

Korean-to-Chinese Machine Translation using Chinese Character as Pivot Clue

Korean-Chinese is a low resource language pair, but Korean and Chinese h...

On the evolution of word usage of classical Chinese poetry

The hierarchy of classical Chinese poetry has been broadly acknowledged ...

A Gap-Based Framework for Chinese Word Segmentation via Very Deep Convolutional Networks

Most previous approaches to Chinese word segmentation can be roughly cla...

Mining Local Gazetteers of Literary Chinese with CRF and Pattern based Methods for Biographical Information in Chinese History

Person names and location names are essential building blocks for identi...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.