Collaging on Internal Representations: An Intuitive Approach for Semantic Transfiguration

11/26/2018
by   Ryohei Suzuki, et al.
0

We present a novel CNN-based image editing method that allows the user to change the semantic information of an image over a user-specified region. Our method makes this possible by combining the idea of manifold projection with spatial conditional batch normalization (sCBN), a version of conditional batch normalization with user-specifiable spatial weight maps. With sCBN and manifold projection, our method lets the user perform (1) spatial class translation that changes the class of an object over an arbitrary region of user's choice, and (2) semantic transplantation that transplants semantic information contained in an arbitrary region of the reference image to an arbitrary region in the target image. These two transformations can be used simultaneously, and can realize a complex composite image-editing task like "change the nose of a beagle to that of a bulldog, and open her mouth." The user can also use our method with intuitive copy-paste-style manipulations. We demonstrate the power of our method on various images. Code will be available at https://github.com/pfnet-research/neural-collage.

READ FULL TEXT

page 2

page 5

page 8

page 9

page 10

page 11

page 15

page 16

research
07/15/2019

Improving the Harmony of the Composite Image by Spatial-Separated Attention Module

Image composition is one of the most important applications in image pro...
research
03/18/2019

Semantic Image Synthesis with Spatially-Adaptive Normalization

We propose spatially-adaptive normalization, a simple but effective laye...
research
11/28/2019

SEAN: Image Synthesis with Semantic Region-Adaptive Normalization

We propose semantic region-adaptive normalization (SEAN), a simple but e...
research
07/27/2016

Instance Normalization: The Missing Ingredient for Fast Stylization

It this paper we revisit the fast stylization method introduced in Ulyan...
research
06/01/2023

Differential Diffusion: Giving Each Pixel Its Strength

Text-based image editing has advanced significantly in recent years. Wit...
research
04/06/2020

Rethinking Spatially-Adaptive Normalization

Spatially-adaptive normalization is remarkably successful recently in co...
research
06/14/2021

Automatically eliminating seam lines with Poisson editing in complex relative radiometric normalization mosaicking scenarios

Relative radiometric normalization (RRN) mosaicking among multiple remot...

Please sign up or login with your details

Forgot password? Click here to reset