DeepAI AI Chat
Log In Sign Up

Tailor: A Prompt-Based Approach to Attribute-Based Controlled Text Generation

04/28/2022
by   Kexin Yang, et al.
0

Attribute-based Controlled Text Generation (CTG) refers to generating sentences that satisfy desirable attributes (e.g., emotions and topics). Existing works often utilize fine-tuning or resort to extra attribute classifiers, yet suffer from storage and inference time increases. To address these concerns, we explore attribute-based CTG in a prompt-based manner. In short, the proposed Tailor represents each attribute as a pre-trained continuous vector (i.e., single-attribute prompt) and guides the generation of a fixed PLM switch to a pre-specified attribute. We experimentally find that these prompts can be simply concatenated as a whole to multi-attribute CTG without any re-training, yet raises problems of fluency decrease and position sensitivity. To this end, Tailor provides a multi-attribute prompt mask and a re-indexing position-ids sequence to bridge the gap between the training (one prompt for each task) and testing stage (concatenating more than one prompt). To further enhance such single-attribute prompt combinations, Tailor also introduces a trainable prompt connector, which can be concatenated with any two single-attribute prompts to multi-attribute text generation. Experiments on 11 attribute-specific generation tasks demonstrate strong performances of Tailor on both single-attribute and multi-attribute CTG, with 0.08% training parameters of a GPT-2.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/04/2019

Plug and Play Language Models: a Simple Approach to Controlled Text Generation

Large transformer-based language models (LMs) trained on huge text corpo...
03/20/2021

Attribute Alignment: Controlling Text Generation from Pre-trained Language Models

Large language models benefit from training with a large amount of unlab...
04/12/2021

FUDGE: Controlled Text Generation With Future Discriminators

We propose Future Discriminators for Generation (FUDGE), a flexible and ...
03/24/2022

Mix and Match: Learning-free Controllable Text Generation using Energy Language Models

Recent work on controlled text generation has either required attribute-...
10/06/2022

A Distributional Lens for Multi-Aspect Controllable Text Generation

Multi-aspect controllable text generation is a more challenging and prac...
06/17/2020

Fine-grained Sentiment Controlled Text Generation

Controlled text generation techniques aim to regulate specific attribute...
06/21/2022

Plug and Play Counterfactual Text Generation for Model Robustness

Generating counterfactual test-cases is an important backbone for testin...