DeepAI AI Chat
Log In Sign Up

Tailor: A Prompt-Based Approach to Attribute-Based Controlled Text Generation

by   Kexin Yang, et al.

Attribute-based Controlled Text Generation (CTG) refers to generating sentences that satisfy desirable attributes (e.g., emotions and topics). Existing works often utilize fine-tuning or resort to extra attribute classifiers, yet suffer from storage and inference time increases. To address these concerns, we explore attribute-based CTG in a prompt-based manner. In short, the proposed Tailor represents each attribute as a pre-trained continuous vector (i.e., single-attribute prompt) and guides the generation of a fixed PLM switch to a pre-specified attribute. We experimentally find that these prompts can be simply concatenated as a whole to multi-attribute CTG without any re-training, yet raises problems of fluency decrease and position sensitivity. To this end, Tailor provides a multi-attribute prompt mask and a re-indexing position-ids sequence to bridge the gap between the training (one prompt for each task) and testing stage (concatenating more than one prompt). To further enhance such single-attribute prompt combinations, Tailor also introduces a trainable prompt connector, which can be concatenated with any two single-attribute prompts to multi-attribute text generation. Experiments on 11 attribute-specific generation tasks demonstrate strong performances of Tailor on both single-attribute and multi-attribute CTG, with 0.08% training parameters of a GPT-2.


page 1

page 2

page 3

page 4


Plug and Play Language Models: a Simple Approach to Controlled Text Generation

Large transformer-based language models (LMs) trained on huge text corpo...

Attribute Alignment: Controlling Text Generation from Pre-trained Language Models

Large language models benefit from training with a large amount of unlab...

FUDGE: Controlled Text Generation With Future Discriminators

We propose Future Discriminators for Generation (FUDGE), a flexible and ...

Mix and Match: Learning-free Controllable Text Generation using Energy Language Models

Recent work on controlled text generation has either required attribute-...

A Distributional Lens for Multi-Aspect Controllable Text Generation

Multi-aspect controllable text generation is a more challenging and prac...

Fine-grained Sentiment Controlled Text Generation

Controlled text generation techniques aim to regulate specific attribute...

Plug and Play Counterfactual Text Generation for Model Robustness

Generating counterfactual test-cases is an important backbone for testin...