Knowledge-Aware Procedural Text Understanding with Multi-Stage Training

09/28/2020
by   Zhihan Zhang, et al.
0

We focus on the task of procedural text understanding, which aims to track entities' states and locations during a natural process. Although recent approaches have achieved substantial progress, they are far behind human performance. Two challenges, difficulty of commonsense reasoning and data insufficiency, still remain unsolved. In this paper, we propose a novel KnOwledge-Aware proceduraL text understAnding (KOALA) model, which leverages external knowledge sources to solve these issues. Specifically, we retrieve informative knowledge triples from ConceptNet and perform knowledge-aware reasoning while tracking the entities. Besides, we employ a multi-stage training schema which fine-tunes the BERT model over unlabeled data collected from Wikipedia before further fine-tuning it on the final model. Experimental results on two procedural text datasets, ProPara and Recipes, verify the effectiveness of the proposed methods, in which our model achieves state-of-the-art performance in comparison to various baselines.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/03/2021

ExBERT: An External Knowledge Enhanced BERT for Natural Language Inference

Neural language representation models such as BERT, pre-trained on large...
research
09/04/2019

KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning

Commonsense reasoning aims to empower machines with the human ability to...
research
05/31/2019

Attention Is (not) All You Need for Commonsense Reasoning

The recently introduced BERT model exhibits strong performance on severa...
research
09/11/2021

Implicit Premise Generation with Discourse-aware Commonsense Knowledge Models

Enthymemes are defined as arguments where a premise or conclusion is lef...
research
08/11/2023

KETM:A Knowledge-Enhanced Text Matching method

Text matching is the task of matching two texts and determining the rela...
research
05/24/2023

Editing Commonsense Knowledge in GPT

Memory editing methods for updating encyclopedic knowledge in transforme...
research
04/03/2022

Pragmatic constraints and pronoun reference disambiguation: the possible and the impossible

Pronoun disambiguation in understanding text and discourse often require...

Please sign up or login with your details

Forgot password? Click here to reset