Confidence-aware Adversarial Learning for Self-supervised Semantic Matching

08/25/2020
by   Shuaiyi Huang, et al.
6

In this paper, we aim to address the challenging task of semantic matching where matching ambiguity is difficult to resolve even with learned deep features. We tackle this problem by taking into account the confidence in predictions and develop a novel refinement strategy to correct partial matching errors. Specifically, we introduce a Confidence-Aware Semantic Matching Network (CAMNet) which instantiates two key ideas of our approach. First, we propose to estimate a dense confidence map for a matching prediction through self-supervised learning. Second, based on the estimated confidence, we refine initial predictions by propagating reliable matching to the rest of locations on the image plane. In addition, we develop a new hybrid loss in which we integrate a semantic alignment loss with a confidence loss, and an adversarial loss that measures the quality of semantic correspondence. We are the first that exploit confidence during refinement to improve semantic matching accuracy and develop an end-to-end self-supervised adversarial learning procedure for the entire matching network. We evaluate our method on two public benchmarks, on which we achieve top performance over the prior state of the art. We will release our source code at https://github.com/ShuaiyiHuang/CAMNet.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 10

10/06/2020

SAFENet: Self-Supervised Monocular Depth Estimation with Semantic-Aware Feature Extraction

Self-supervised monocular depth estimation has emerged as a promising me...
09/08/2019

Dynamic Context Correspondence Network for Semantic Alignment

Establishing semantic correspondence is a core problem in computer visio...
09/28/2021

PDC-Net+: Enhanced Probabilistic Dense Correspondence Network

Establishing robust and accurate correspondences between a pair of image...
12/31/2016

Improved Stereo Matching with Constant Highway Networks and Reflective Confidence Learning

We present an improved three-step pipeline for the stereo matching probl...
07/23/2019

ChromaGAN: An Adversarial Approach for Picture Colorization

The colorization of grayscale images is an ill-posed problem, with multi...
01/05/2021

Learning Accurate Dense Correspondences and When to Trust Them

Establishing dense correspondences between a pair of images is an import...
05/28/2018

Confidence Prediction for Lexicon-Free OCR

Having a reliable accuracy score is crucial for real world applications ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.