Semantic Editing On Segmentation Map Via Multi-Expansion Loss

by   Jianfeng He, et al.

Semantic editing on segmentation map has been proposed as an intermediate interface for image generation, because it provides flexible and strong assistance in various image generation tasks. This paper aims to improve quality of edited segmentation map conditioned on semantic inputs. Even though recent studies apply global and local adversarial losses extensively to generate images for higher image quality, we find that they suffer from the misalignment of the boundary area in the mask area. To address this, we propose MExGAN for semantic editing on segmentation map, which uses a novel Multi-Expansion (MEx) loss implemented by adversarial losses on MEx areas. Each MEx area has the mask area of the generation as the majority and the boundary of original context as the minority. To boost convenience and stability of MEx loss, we further propose an Approximated MEx (A-MEx) loss. Besides, in contrast to previous model that builds training data for semantic editing on segmentation map with part of the whole image, which leads to model performance degradation, MExGAN applies the whole image to build the training data. Extensive experiments on semantic editing on segmentation map and natural image inpainting show competitive results on four datasets.


page 1

page 4

page 6

page 12

page 13

page 14

page 15

page 16


DiffEdit: Diffusion-based semantic image editing with mask guidance

Image generation has recently seen tremendous advances, with diffusion m...

SESAME: Semantic Editing of Scenes by Adding, Manipulating or Erasing Objects

Recent advances in image generation gave rise to powerful tools for sema...

Automatically eliminating seam lines with Poisson editing in complex relative radiometric normalization mosaicking scenarios

Relative radiometric normalization (RRN) mosaicking among multiple remot...

Designing a Better Asymmetric VQGAN for StableDiffusion

StableDiffusion is a revolutionary text-to-image generator that is causi...

From Sparse to Precise: A Practical Editing Approach for Intracardiac Echocardiography Segmentation

Accurate and safe catheter ablation procedures for patients with atrial ...

GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generation

Segmenting an image into its parts is a frequent preprocess for high-lev...

Alpha Matte Generation from Single Input for Portrait Matting

Portrait matting is an important research problem with a wide range of a...

Please sign up or login with your details

Forgot password? Click here to reset