KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning

by   Ye Liu, et al.

Generative commonsense reasoning which aims to empower machines to generate sentences with the capacity of reasoning over a set of concepts is a critical bottleneck for text generation. Even the state-of-the-art pre-trained language generation models struggle at this task and often produce implausible and anomalous sentences. One reason is that they rarely consider incorporating the knowledge graph which can provide rich relational information among the commonsense concepts. To promote the ability of commonsense reasoning for text generation, we propose a novel knowledge graph augmented pre-trained language generation model KG-BART, which encompasses the complex relations of concepts through the knowledge graph and produces more logical and natural sentences as output. Moreover, KG-BART can leverage the graph attention to aggregate the rich concept semantics that enhances the model generalization on unseen concept sets. Experiments on benchmark CommonGen dataset verify the effectiveness of our proposed approach by comparing with several strong pre-trained language generation models, particularly KG-BART outperforms BART by 5.80, 4.60, in terms of BLEU-3, 4. Moreover, we also show that the generated context by our model can work as background scenarios to benefit downstream commonsense QA tasks.


page 3

page 7


Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph

Despite the success of generative pre-trained language models on a serie...

Contextualized Scene Imagination for Generative Commonsense Reasoning

Humans use natural language to compose common concepts from their enviro...

CommonGen: A Constrained Text Generation Dataset Towards Generative Commonsense Reasoning

Rational humans can generate sentences that cover a certain set of conce...

Inferring Commonsense Explanations as Prompts for Future Event Generation

Future Event Generation aims to generate fluent and reasonable future ev...

KGR^4: Retrieval, Retrospect, Refine and Rethink for Commonsense Generation

Generative commonsense reasoning requires machines to generate sentences...

Lexically-constrained Text Generation through Commonsense Knowledge Extraction and Injection

Conditional text generation has been a challenging task that is yet to s...

Revisiting Generative Commonsense Reasoning: A Pre-Ordering Approach

Pre-trained models (PTMs) have lead to great improvements in natural lan...