DiffMatch: Diffusion Model for Dense Matching

05/30/2023
by   Jisu Nam, et al.
0

The objective for establishing dense correspondence between paired images consists of two terms: a data term and a prior term. While conventional techniques focused on defining hand-designed prior terms, which are difficult to formulate, recent approaches have focused on learning the data term with deep neural networks without explicitly modeling the prior, assuming that the model itself has the capacity to learn an optimal prior from a large-scale dataset. The performance improvement was obvious, however, they often fail to address inherent ambiguities of matching, such as textureless regions, repetitive patterns, and large displacements. To address this, we propose DiffMatch, a novel conditional diffusion-based framework designed to explicitly model both the data and prior terms. Unlike previous approaches, this is accomplished by leveraging a conditional denoising diffusion model. DiffMatch consists of two main components: conditional denoising diffusion module and cost injection module. We stabilize the training process and reduce memory usage with a stage-wise training strategy. Furthermore, to boost performance, we introduce an inference technique that finds a better path to the accurate matching field. Our experimental results demonstrate significant performance improvements of our method over existing approaches, and the ablation studies validate our design choices along with the effectiveness of each component. Project page is available at https://ku-cvlab.github.io/DiffMatch/.

READ FULL TEXT

page 1

page 5

page 8

page 16

page 17

page 18

page 19

page 20

research
08/01/2023

Diffusion Model for Camouflaged Object Detection

Camouflaged object detection is a challenging task that aims to identify...
research
06/06/2021

Deep Matching Prior: Test-Time Optimization for Dense Correspondence

Conventional techniques to establish dense correspondences across visual...
research
08/13/2023

Unsupervised Image Denoising in Real-World Scenarios via Self-Collaboration Parallel Generative Adversarial Branches

Deep learning methods have shown remarkable performance in image denoisi...
research
06/07/2023

On the Design Fundamentals of Diffusion Models: A Survey

Diffusion models are generative models, which gradually add and remove n...
research
03/30/2023

PMatch: Paired Masked Image Modeling for Dense Geometric Matching

Dense geometric matching determines the dense pixel-wise correspondence ...
research
09/03/2023

NADiffuSE: Noise-aware Diffusion-based Model for Speech Enhancement

The goal of speech enhancement (SE) is to eliminate the background inter...
research
05/22/2023

Hierarchical Integration Diffusion Model for Realistic Image Deblurring

Diffusion models (DMs) have recently been introduced in image deblurring...

Please sign up or login with your details

Forgot password? Click here to reset