Fast and Accurate Neural CRF Constituency Parsing

by   Yu Zhang, et al.

Estimating probability distribution is one of the core issues in the NLP field. However, in both deep learning (DL) and pre-DL eras, unlike the vast applications of linear-chain CRF in sequence labeling tasks, very few works have applied tree-structure CRF to constituency parsing, mainly due to the complexity and inefficiency of the inside-outside algorithm. This work presents a fast and accurate neural CRF constituency parser. The key idea is to batchify the inside algorithm for loss computation by direct large tensor operations on GPU, and meanwhile avoid the outside algorithm for gradient computation via efficient back-propagation. We also propose a simple two-stage bracketing-then-labeling parsing approach to improve efficiency further. To improve the parsing performance, inspired by recent progress in dependency parsing, we introduce a new scoring architecture based on boundary representation and biaffine attention, and a beneficial dropout strategy. Experiments on PTB, CTB5.1, and CTB7 show that our two-stage CRF parser achieves new state-of-the-art performance on both settings of w/o and w/ BERT, and can parse over 1,000 sentences per second. We release our code at



There are no comments yet.


page 1

page 2

page 3

page 4


Efficient Second-Order TreeCRF for Neural Dependency Parsing

In the deep learning (DL) era, parsing models are extremely simplified w...

Fast and Accurate Span-based Semantic Role Labeling as Graph Parsing

Currently, BIO-based and tuple-based approaches perform quite well on th...

Neural CRF Parsing

This paper describes a parsing model that combines the exact dynamic pro...

Fast and Accurate Sequence Labeling with Approximate Inference Network

The linear-chain Conditional Random Field (CRF) model is one of the most...

Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Autoencoders

We introduce deep inside-outside recursive autoencoders (DIORA), a fully...

CRF Autoencoder for Unsupervised Dependency Parsing

Unsupervised dependency parsing, which tries to discover linguistic depe...

A Fast Unified Model for Parsing and Sentence Understanding

Tree-structured neural networks exploit valuable syntactic parse informa...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.