Robust Chinese Word Segmentation with Contextualized Word Representations

01/17/2019
by   Yung-Sung Chuang, et al.
0

In recent years, after the neural-network-based method was proposed, the accuracy of the Chinese word segmentation task has made great progress. However, when dealing with out-of-vocabulary words, there is still a large error rate. We used a simple bidirectional LSTM architecture and a large-scale pretrained language model to generate high-quality contextualize character representations, which successfully reduced the weakness of the ambiguous meanings of each Chinese character that widely appears in Chinese characters, and hence effectively reduced OOV error rate. State-of-the-art performance is achieved on many datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/16/2017

Learning Chinese Word Representations From Glyphs Of Characters

In this paper, we propose new methods to learn Chinese word representati...
research
02/27/2018

A Hybrid Word-Character Model for Abstractive Summarization

Abstractive summarization is the popular research topic nowadays. Due to...
research
08/20/2018

State-of-the-art Chinese Word Segmentation with Bi-LSTMs

A wide variety of neural-network architectures have been proposed for th...
research
01/22/2022

Chinese Word Segmentation with Heterogeneous Graph Neural Network

In recent years, deep learning has achieved significant success in the C...
research
04/05/2017

Character-based Joint Segmentation and POS Tagging for Chinese using Bidirectional RNN-CRF

We present a character-based model for joint segmentation and POS taggin...
research
05/01/2018

Nugget Proposal Networks for Chinese Event Detection

Neural network based models commonly regard event detection as a word-wi...
research
12/11/2018

Hyperbolic Deep Learning for Chinese Natural Language Understanding

Recently hyperbolic geometry has proven to be effective in building embe...

Please sign up or login with your details

Forgot password? Click here to reset