Open-Ended Content-Style Recombination Via Leakage Filtering

09/28/2018
by   Karl Ridgeway, et al.
0

We consider visual domains in which a class label specifies the content of an image, and class-irrelevant properties that differentiate instances constitute the style. We present a domain-independent method that permits the open-ended recombination of style of one image with the content of another. Open ended simply means that the method generalizes to style and content not present in the training data. The method starts by constructing a content embedding using an existing deep metric-learning technique. This trained content encoder is incorporated into a variational autoencoder (VAE), paired with a to-be-trained style encoder. The VAE reconstruction loss alone is inadequate to ensure a decomposition of the latent representation into style and content. Our method thus includes an auxiliary loss, leakage filtering, which ensures that no style information remaining in the content representation is used for reconstruction and vice versa. We synthesize novel images by decoding the style representation obtained from one image with the content representation from another. Using this method for data-set augmentation, we obtain state-of-the-art performance on few-shot learning tasks.

READ FULL TEXT
research
03/20/2022

Partitioning Image Representation in Contrastive Learning

In contrastive learning in the image domain, the anchor and positive sam...
research
02/21/2021

Rethinking Content and Style: Exploring Bias for Unsupervised Disentanglement

Content and style (C-S) disentanglement intends to decompose the underly...
research
02/15/2022

Disentangling Domain and Content

Many real-world datasets can be divided into groups according to certain...
research
08/04/2021

Information Sieve: Content Leakage Reduction in End-to-End Prosody For Expressive Speech Synthesis

Expressive neural text-to-speech (TTS) systems incorporate a style encod...
research
10/02/2016

Deep Feature Consistent Variational Autoencoder

We present a novel method for constructing Variational Autoencoder (VAE)...
research
07/07/2022

Harnessing Out-Of-Distribution Examples via Augmenting Content and Style

Machine learning models are vulnerable to Out-Of-Distribution (OOD) exam...
research
01/14/2020

Adversarial Disentanglement with Grouped Observations

We consider the disentanglement of the representations of the relevant a...

Please sign up or login with your details

Forgot password? Click here to reset