DeepAI AI Chat
Log In Sign Up

MIDMs: Matching Interleaved Diffusion Models for Exemplar-based Image Translation

by   Junyoung Seo, et al.

We present a novel method for exemplar-based image translation, called matching interleaved diffusion models (MIDMs). Most existing methods for this task were formulated as GAN-based matching-then-generation framework. However, in this framework, matching errors induced by the difficulty of semantic matching across cross-domain, e.g., sketch and photo, can be easily propagated to the generation step, which in turn leads to degenerated results. Motivated by the recent success of diffusion models overcoming the shortcomings of GANs, we incorporate the diffusion models to overcome these limitations. Specifically, we formulate a diffusion-based matching-and-generation framework that interleaves cross-domain matching and diffusion steps in the latent space by iteratively feeding the intermediate warp into the noising process and denoising it to generate a translated image. In addition, to improve the reliability of the diffusion process, we design a confidence-aware process using cycle-consistency to consider only confident regions during translation. Experimental results show that our MIDMs generate more plausible images than state-of-the-art methods.


page 5

page 6

page 12

page 14

page 15

page 16

page 17

page 18


Cross-domain Correspondence Learning for Exemplar-based Image Translation

We present a general framework for exemplar-based image translation, whi...

DiffusionCLIP: Text-guided Image Manipulation Using Diffusion Models

Diffusion models are recent generative models that have shown great succ...

CFFT-GAN: Cross-domain Feature Fusion Transformer for Exemplar-based Image Translation

Exemplar-based image translation refers to the task of generating images...

Diffusion-based Image Translation using Disentangled Style and Content Representation

Diffusion-based image translation guided by semantic texts or a single t...

Cross-domain Compositing with Pretrained Diffusion Models

Diffusion models have enabled high-quality, conditional image editing ca...

XNet: GAN Latent Space Constraints

Recent GAN-based architectures have been able to deliver impressive perf...

Image Translation by Latent Union of Subspaces for Cross-Domain Plaque Detection

Calcified plaque in the aorta and pelvic arteries is associated with cor...