How Sequence-to-Sequence Models Perceive Language Styles?

by   Ruozi Huang, et al.
FUDAN University

Style is ubiquitous in our daily language uses, while what is language style to learning machines? In this paper, by exploiting the second-order statistics of semantic vectors of different corpora, we present a novel perspective on this question via style matrix, i.e. the covariance matrix of semantic vectors, and explain for the first time how Sequence-to-Sequence models encode style information innately in its semantic vectors. As an application, we devise a learning-free text style transfer algorithm, which explicitly constructs a pair of transfer operators from the style matrices for style transfer. Moreover, our algorithm is also observed to be flexible enough to transfer out-of-domain sentences. Extensive experimental evidence justifies the informativeness of style matrix and the competitive performance of our proposed style transfer algorithm with the state-of-the-art methods.


page 1

page 2

page 3

page 4


Learning Linear Transformations for Fast Arbitrary Style Transfer

Given a random pair of images, an arbitrary style transfer method extrac...

Replacing Language Model for Style Transfer

We introduce replacing language model (RLM), a sequence-to-sequence lang...

Multimodal Style Transfer via Graph Cuts

An assumption widely used in recent neural style transfer methods is tha...

Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer

Scarcity of parallel data causes formality style transfer models to have...

Grammatical Error Correction and Style Transfer via Zero-shot Monolingual Translation

Both grammatical error correction and text style transfer can be viewed ...

Multi-Pair Text Style Transfer on Unbalanced Data

Text-style transfer aims to convert text given in one domain into anothe...

Large-scale Hierarchical Alignment for Author Style Transfer

We propose a simple method for extracting pseudo-parallel monolingual se...

Please sign up or login with your details

Forgot password? Click here to reset