Deep Stacking Networks for Low-Resource Chinese Word Segmentation with Transfer Learning

11/04/2017
by   Jingjing Xu, et al.
0

In recent years, neural networks have proven to be effective in Chinese word segmentation. However, this promising performance relies on large-scale training data. Neural networks with conventional architectures cannot achieve the desired results in low-resource datasets due to the lack of labelled training data. In this paper, we propose a deep stacking framework to improve the performance on word segmentation tasks with insufficient data by integrating datasets from diverse domains. Our framework consists of two parts, domain-based models and deep stacking networks. The domain-based models are used to learn knowledge from different datasets. The deep stacking networks are designed to integrate domain-based models. To reduce model conflicts, we innovatively add communication paths among models and design various structures of deep stacking networks, including Gaussian-based Stacking Networks, Concatenate-based Stacking Networks, Sequence-based Stacking Networks and Tree-based Stacking Networks. We conduct experiments on six low-resource datasets from various domains. Our proposed framework shows significant performance improvements on all datasets compared with several strong baselines.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/15/2017

Transfer Deep Learning for Low-Resource Chinese Word Segmentation with a Novel Neural Network

Recent studies have shown effectiveness in using neural networks for Chi...
research
08/28/2018

Deriving Machine Attention from Human Rationales

Attention-based models are successful when trained on large amounts of d...
research
08/12/2020

Approaching Neural Chinese Word Segmentation as a Low-Resource Machine Translation Task

Supervised Chinese word segmentation has been widely approached as seque...
research
10/31/2022

Mining Word Boundaries in Speech as Naturally Annotated Word Segmentation Data

Chinese word segmentation (CWS) models have achieved very high performan...
research
07/10/2018

Deep Learning on Low-Resource Datasets

In training a deep learning system to perform audio transcription, two p...
research
07/10/2018

Deep Learning for Audio Transcription on Low-Resource Datasets

In training a deep learning system to perform audio transcription, two p...
research
11/17/2021

Green CWS: Extreme Distillation and Efficient Decode Method Towards Industrial Application

Benefiting from the strong ability of the pre-trained model, the researc...

Please sign up or login with your details

Forgot password? Click here to reset