DeepAI AI Chat
Log In Sign Up

Zooming Network

by   Yukun Yan, et al.
Tsinghua University

Structural information is important in natural language understanding. Although some current neural net-based models have a limited ability to take local syntactic information, they fail to use high-level and large-scale structures of documents. This information is valuable for text understanding since it contains the author's strategy to express information, in building an effective representation and forming appropriate output modes. We propose a neural net-based model, Zooming Network, capable of representing and leveraging text structure of long document and developing its own analyzing rhythm to extract critical information. Generally, ZN consists of an encoding neural net that can build a hierarchical representation of a document, and an interpreting neural model that can read the information at multi-levels and issuing labeling actions through a policy-net. Our model is trained with a hybrid paradigm of supervised learning (distinguishing right and wrong decision) and reinforcement learning (determining the goodness among multiple right paths). We applied the proposed model to long text sequence labeling tasks, with performance exceeding baseline model (biLSTM-crf) by 10 F1-measure.


Event Identification as a Decision Process with Non-linear Representation of Text

We propose scale-free Identifier Network(sfIN), a novel model for event ...

Topic-Aware Encoding for Extractive Summarization

Document summarization provides an instrument for faster understanding t...

ReadOnce Transformers: Reusable Representations of Text for Transformers

While large-scale language models are extremely effective when directly ...

Doc2Dict: Information Extraction as Text Generation

Typically, information extraction (IE) requires a pipeline approach: fir...

Hierarchical RNN for Information Extraction from Lawsuit Documents

Every lawsuit document contains the information about the party's claim,...

Hierarchical Encoders for Modeling and Interpreting Screenplays

While natural language understanding of long-form documents is still an ...

Interpreting Audiograms with Multi-stage Neural Networks

Audiograms are a particular type of line charts representing individuals...