Unsupervised Text Style Transfer with Padded Masked Language Models

10/02/2020
by   Eric Malmi, et al.
0

We propose Masker, an unsupervised text-editing method for style transfer. To tackle cases when no parallel source-target pairs are available, we train masked language models (MLMs) for both the source and the target domain. Then we find the text spans where the two models disagree the most in terms of likelihood. This allows us to identify the source tokens to delete to transform the source text to match the style of the target domain. The deleted tokens are replaced with the target MLM, and by using a padded MLM variant, we avoid having to predetermine the number of inserted tokens. Our experiments on sentence fusion and sentiment transfer demonstrate that Masker performs competitively in a fully unsupervised setting. Moreover, in low-resource settings, it improves supervised methods' accuracy by over 10 percentage points when pre-training them on silver training data generated by Masker.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/18/2021

LEWIS: Levenshtein Editing for Unsupervised Text Style Transfer

Many types of text style transfer can be achieved with only small, preci...
research
10/12/2022

On Text Style Transfer via Style Masked Language Models

Text Style Transfer (TST) is performable through approaches such as late...
research
05/19/2023

Reducing Sequence Length by Predicting Edit Operations with Large Language Models

Large Language Models (LLMs) have demonstrated remarkable performance in...
research
05/30/2018

Unsupervised Text Style Transfer using Language Models as Discriminators

Binary classifiers are often employed as discriminators in GAN-based uns...
research
05/09/2022

So Different Yet So Alike! Constrained Unsupervised Text Style Transfer

Automatic transfer of text between domains has become popular in recent ...
research
09/09/2021

Generic resources are what you need: Style transfer tasks without task-specific parallel training data

Style transfer aims to rewrite a source text in a different target style...
research
10/10/2020

Semi-supervised Formality Style Transfer using Language Model Discriminator and Mutual Information Maximization

Formality style transfer is the task of converting informal sentences to...

Please sign up or login with your details

Forgot password? Click here to reset