Log In Sign Up

Lexically-constrained Text Generation through Commonsense Knowledge Extraction and Injection

by   Yikang Li, et al.

Conditional text generation has been a challenging task that is yet to see human-level performance from state-of-the-art models. In this work, we specifically focus on the Commongen benchmark, wherein the aim is to generate a plausible sentence for a given set of input concepts. Despite advances in other tasks, large pre-trained language models that are fine-tuned on this dataset often produce sentences that are syntactically correct but qualitatively deviate from a human understanding of common sense. Furthermore, generated sequences are unable to fulfill such lexical requirements as matching part-of-speech and full concept coverage. In this paper, we explore how commonsense knowledge graphs can enhance model performance, with respect to commonsense reasoning and lexically-constrained decoding. We propose strategies for enhancing the semantic correctness of the generated text, which we accomplish through: extracting commonsense relations from Conceptnet, injecting these relations into the Unified Language Model (UniLM) through attention mechanisms, and enforcing the aforementioned lexical requirements through output constraints. By performing several ablations, we find that commonsense injection enables the generation of sentences that are more aligned with human understanding, while remaining compliant with lexical requirements.


page 1

page 2

page 3

page 4


Contextualized Scene Imagination for Generative Commonsense Reasoning

Humans use natural language to compose common concepts from their enviro...

KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning

Generative commonsense reasoning which aims to empower machines to gener...

On-the-Fly Attention Modularization for Neural Generation

Despite considerable advancements with deep neural language models (LMs)...

MERMAID: Metaphor Generation with Symbolism and Discriminative Decoding

Generating metaphors is a challenging task as it requires a proper under...

Guided Generation of Cause and Effect

We present a conditional text generation framework that posits sententia...

Constrained Text Generation with Global Guidance – Case Study on CommonGen

This paper studies constrained text generation, which is to generate sen...

An Enhanced Knowledge Injection Model for Commonsense Generation

Commonsense generation aims at generating plausible everyday scenario de...