Semi-supervised Formality Style Transfer using Language Model Discriminator and Mutual Information Maximization

10/10/2020
by   Kunal Chawla, et al.
0

Formality style transfer is the task of converting informal sentences to grammatically-correct formal sentences, which can be used to improve performance of many downstream NLP tasks. In this work, we propose a semi-supervised formality style transfer model that utilizes a language model-based discriminator to maximize the likelihood of the output sentence being formal, which allows us to use maximization of token-level conditional probabilities for training. We further propose to maximize mutual information between source and target styles as our training objective instead of maximizing the regular likelihood that often leads to repetitive and trivial generated responses. Experiments showed that our model outperformed previous state-of-the-art baselines significantly in terms of both automated metrics and human judgement. We further generalized our model to unsupervised text style transfer task, and achieved significant improvements on two benchmark sentiment style transfer datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/14/2022

Replacing Language Model for Style Transfer

We introduce replacing language model (RLM), a sequence-to-sequence lang...
research
05/30/2018

Unsupervised Text Style Transfer using Language Models as Discriminators

Binary classifiers are often employed as discriminators in GAN-based uns...
research
03/26/2019

Reinforcement Learning Based Text Style Transfer without Parallel Training Corpus

Text style transfer rephrases a text from a source style (e.g., informal...
research
03/25/2022

Semi-Supervised Formality Style Transfer with Consistency Training

Formality style transfer (FST) is a task that involves paraphrasing an i...
research
10/02/2020

Unsupervised Text Style Transfer with Padded Masked Language Models

We propose Masker, an unsupervised text-editing method for style transfe...
research
09/14/2023

Speech-to-Speech Translation with Discrete-Unit-Based Style Transfer

Direct speech-to-speech translation (S2ST) with discrete self-supervised...
research
08/13/2018

What is wrong with style transfer for texts?

A number of recent machine learning papers work with an automated style ...

Please sign up or login with your details

Forgot password? Click here to reset