Parallax Attention for Unsupervised Stereo Correspondence Learning

by   Longguang Wang, et al.

Stereo image pairs encode 3D scene cues into stereo correspondences between the left and right images. To exploit 3D cues within stereo images, recent CNN based methods commonly use cost volume techniques to capture stereo correspondence over large disparities. However, since disparities can vary significantly for stereo cameras with different baselines, focal lengths and resolutions, the fixed maximum disparity used in cost volume techniques hinders them to handle different stereo image pairs with large disparity variations. In this paper, we propose a generic parallax-attention mechanism (PAM) to capture stereo correspondence regardless of disparity variations. Our PAM integrates epipolar constraints with attention mechanism to calculate feature similarities along the epipolar line to capture stereo correspondence. Based on our PAM, we propose a parallax-attention stereo matching network (PASMnet) and a parallax-attention stereo image super-resolution network (PASSRnet) for stereo matching and stereo image super-resolution tasks. Moreover, we introduce a new and large-scale dataset named Flickr1024 for stereo image super-resolution. Experimental results show that our PAM is generic and can effectively learn stereo correspondence under large disparity variations in an unsupervised manner. Comparative results show that our PASMnet and PASSRnet achieve the state-of-the-art performance.


page 4

page 5

page 6

page 9

page 11

page 12

page 15

page 18


Learning Parallax Attention for Stereo Image Super-Resolution

Stereo image pairs can be used to improve the performance of super-resol...

Cross-MPI: Cross-scale Stereo for Image Super-Resolution using Multiplane Images

The combination of various cameras is enriching the way of computational...

Stereo Superpixel Segmentation Via Decoupled Dynamic Spatial-Embedding Fusion Network

Stereo superpixel segmentation aims at grouping the discretizing pixels ...

Degradation-agnostic Correspondence from Resolution-asymmetric Stereo

In this paper, we study the problem of stereo matching from a pair of im...

Multi-Resolution Factor Graph Based Stereo Correspondence Algorithm

A dense depth-map of a scene at an arbitrary view orientation can be est...

Rectifying homographies for stereo vision: analytical solution for minimal distortion

Stereo rectification is the determination of two image transformations (...

Feedback Network for Mutually Boosted Stereo Image Super-Resolution and Disparity Estimation

Under stereo settings, the problem of image super-resolution (SR) and di...

Please sign up or login with your details

Forgot password? Click here to reset