MixSyn: Learning Composition and Style for Multi-Source Image Synthesis

11/24/2021
by   Ilke Demir, et al.
4

Synthetic images created by generative models increase in quality and expressiveness as newer models utilize larger datasets and novel architectures. Although this photorealism is a positive side-effect from a creative standpoint, it becomes problematic when such generative models are used for impersonation without consent. Most of these approaches are built on the partial transfer between source and target pairs, or they generate completely new samples based on an ideal distribution, still resembling the closest real sample in the dataset. We propose MixSyn (read as " mixin' ") for learning novel fuzzy compositions from multiple sources and creating novel images as a mix of image regions corresponding to the compositions. MixSyn not only combines uncorrelated regions from multiple source masks into a coherent semantic composition, but also generates mask-aware high quality reconstructions of non-existing images. We compare MixSyn to state-of-the-art single-source sequential generation and collage generation approaches in terms of quality, diversity, realism, and expressive power; while also showcasing interactive synthesis, mix match, and edit propagation tasks, with no mask dependency.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 6

page 7

page 8

page 9

research
06/08/2022

Accelerating Score-based Generative Models for High-Resolution Image Synthesis

Score-based generative models (SGMs) have recently emerged as a promisin...
research
03/19/2023

Less is More: Unsupervised Mask-guided Annotated CT Image Synthesis with Minimum Manual Segmentations

As a pragmatic data augmentation tool, data synthesis has generally retu...
research
11/09/2015

Generating Images from Captions with Attention

Motivated by the recent progress in generative models, we introduce a mo...
research
11/28/2019

SEAN: Image Synthesis with Semantic Region-Adaptive Normalization

We propose semantic region-adaptive normalization (SEAN), a simple but e...
research
06/17/2022

HairFIT: Pose-Invariant Hairstyle Transfer via Flow-based Hair Alignment and Semantic-Region-Aware Inpainting

Hairstyle transfer is the task of modifying a source hairstyle to a targ...
research
04/05/2023

Few-shot Semantic Image Synthesis with Class Affinity Transfer

Semantic image synthesis aims to generate photo realistic images given a...

Please sign up or login with your details

Forgot password? Click here to reset