-
Example-Guided Scene Image Synthesis using Masked Spatial-Channel Attention and Patch-Based Self-Supervision
Example-guided image synthesis has been recently attempted to synthesize...
read it
-
Example-Guided Style Consistent Image Synthesis from Semantic Labeling
Example-guided image synthesis aims to synthesize an image from a semant...
read it
-
Dual Attention GANs for Semantic Image Synthesis
In this paper, we focus on the semantic image synthesis task that aims a...
read it
-
Curiosity-driven 3D Scene Structure from Single-image Self-supervision
Previous work has demonstrated learning isolated 3D objects (voxel grids...
read it
-
Attention-aware Multi-stroke Style Transfer
Neural style transfer has drawn considerable attention from both academi...
read it
-
Editing in Style: Uncovering the Local Semantics of GANs
While the quality of GAN image synthesis has improved tremendously in re...
read it
-
Sketch-Guided Scenery Image Outpainting
The outpainting results produced by existing approaches are often too ra...
read it
Example-Guided Image Synthesis across Arbitrary Scenes using Masked Spatial-Channel Attention and Self-Supervision
Example-guided image synthesis has recently been attempted to synthesize an image from a semantic label map and an exemplary image. In the task, the additional exemplar image provides the style guidance that controls the appearance of the synthesized output. Despite the controllability advantage, the existing models are designed on datasets with specific and roughly aligned objects. In this paper, we tackle a more challenging and general task, where the exemplar is an arbitrary scene image that is semantically different from the given label map. To this end, we first propose a Masked Spatial-Channel Attention (MSCA) module which models the correspondence between two arbitrary scenes via efficient decoupled attention. Next, we propose an end-to-end network for joint global and local feature alignment and synthesis. Finally, we propose a novel self-supervision task to enable training. Experiments on the large-scale and more diverse COCO-stuff dataset show significant improvements over the existing methods. Moreover, our approach provides interpretability and can be readily extended to other content manipulation tasks including style and spatial interpolation or extrapolation.
READ FULL TEXT
Comments
There are no comments yet.