In safety-critical applications such as autonomous driving, it is important to accurately perceive the depth of obstacles with low latency. Towards that end, we explore the problem of real-time stereo matching on high-res imagery.
LiDAR vs Stereo: LiDAR is the common choice for outdoor depth sensing . However, they are fundamentally limited in spatial density, particularly for long-range sensing. Only so many beams and detectors can be packed together before cross-talk occurs. In principle, one can increase density by scanning slowly, but this introduces latency and rolling shutter effects that can be devastating for dynamic scenes. Density of sensor measurements is crucial for recognizing and segmenting objects at range. High-res, global shutter stereo has the potential to address these limitations.
Why high-res? It is widely recognized that stereo is unreliable for long-range depth sensing 
: the metric error in estimated depthof a triangulation-based stereo system with baseline , focal length , and pixel matching error can be written as . Hence depth error increases quadratically with depth, implying that stereo will provide unstable far-field depth estimates. But this is important for navigation at even moderate speeds - (see the supplement for stopping distance profiles). Though one can attempt to tweak the other factors to reduce the error, higher-resolution (larger ) appear to be the most promising avenue: innovations in CMOS and CCD sensor technology have allowed for mass-market, low-cost solutions for high-resolution cameras.
Challenges: Although high-res stereo matching is desirable, there are several practical challenges: optimization-based stereo methods are accurate but cannot scale to high-resolution with regard to both running time and memory overhead. When applied to down-scaled images, these methods run faster, but gives blurry results and inaccurate disparity estimates for the far-field. Recent “deep” stereo methods perform well on low-resolution benchmarks [5, 11, 16, 21, 38], while failing to produce SOTA results on high-res benchmarks . This is likely due to: 1) Their architectures are not efficiently designed to operate on high-resolution images. 2) They do not have enough high-resolution training data.
Approach: We propose an end-to-end framework that efficiently searches for correspondences over hierarchies. Our model reasons in a coarse-to-fine-manner, inspired by classic work on correspondence estimation in stereo and optical flow [1, 18, 39]. Coarse resolution images are used to estimate large disparities, which are then used to bias/pre-warp fine-scale disparity estimates. Though quite efficient, coarse-to-fine methods struggle to match thin structures that “disappear” at coarse resolutions . Instead, our model computes a high-res encoder feature that is processed with coarse-to-fine (decoder) feature volumes that gradually increase in resolution. Crucially, the initial coarse volume can generate rough estimates of large-disparity objects before the full pipeline is finished. This allows our network to generate reports of closeby objects on-demand, which can be crucial for real-time navigation at speed.
Data: High-res stereo efforts suffer from the lack of benchmark data, for both training and evaluation. We have collected two datasets of high-res rectified stereo pairs, consisting of real-data from an autonomous vehicle and synthetic data from an urban simulator. Interestingly, we show that synthetic data is a valuable tool for training deep stereo networks, particularly for high-res disparity estimation. Real-world calibration and rectification become challenging at high-res, and introducing realistic calibration errors through data-augmentation is helpful for training.
Our main contributions are as follows:
We propose a hierarchical stereo matching architecture that processes high-resolution images while being able to perform on-demand computation in real-time.
We synthesize a high-res stereo dataset for training and collect a real high-res dataset for testing.
We introduce a set of stereo augmentation techniques to improve the model’s robustness to calibration errors, exposure changes, and camera occlusions.
We achieve SOTA accuracy on Middlebury and KITTI while running significantly faster than the prior art.
2 Related Work
Stereo matching is a classic task in computer vision. Traditional methods take rectified image pairs as input (though extensions to multiview stereo exist ), extract local descriptors at candidate patches [35, 36] and then build up -dimensional cost volumes across corresponding epipolar scanlines . To ensure global ordering and consistency constraints, global optimization techniques are applied with a considerable cost of time and memory, which also poses limitations on scale [10, 14].
Efficient high-resolution stereo: To mitigate this problem, SGM  and ELAS  describe efficient matching algorithms that allow for FPS on -megapixel images with a strong performance. However, both methods struggle to scale to -megapixel images: for example, SGM requires GB to process a disparity search range of 700px, and surprisingly, performance drops when compared to a variant that processes lower-res inputs . While other work has also explored efficient high-res processing [15, 30], they do not appear to meet the accuracy and speed requirements for real-time sensing in autonomous driving.
Deep stereo matching: Deep networks tuned for stereo estimation can take advantage of large-scale annotated data. They now produce SOTA performance on several stereo benchmarks, though with considerable use of memory and time. Zbontar et al. and Luo et al. [19, 40] use siamese networks to extract patch-wise features, which are then processed in a traditional cost volume by classic post-processing. Some recent methods [5, 12, 13, 16, 20, 23] replace post-processing with 2D/3D convolutions applied to the cost volume, producing SOTA performance on the KITTI benchmark. However surprisingly, none of them outperform traditional methods on Middlebury, possibly due to 1) memory constraints and 2) lack of training data. Although one may run low-res models on crops of high-res images and stitch together the predictions, however, challenges are that: 1) crop borders can be challenging to match; 2) contextual information may not be well-utilized; and 3) most importantly, this dramatically increases run-time latency.
To our knowledge, we are the first to successfully address these issues and apply deep networks on high-res stereo matching: We propose an efficient hierarchical stereo matching architecture to address the efficiency issue, and leverage high-res synthetic data as well as novel augmentation techniques to overcome data scarcity.
Coarse-to-fine CNNs: Coarse-to-fine design in CNNs dates back to FCN and U-Net [17, 25], which leverages multi-scale features and aggregate coarse-to-fine predictions to improve semantic segmentation. DispNet-based architectures [20, 23] adopts an encoder-decoder scheme with skip-connections, which compute multi-scale cost volumes by correlation and apply 2D convolution/up-convolutions to refine predictions from coarse-to-fine. Recently, PWCNet  uses a coarse-to-fine architecture to warp features and achieves SOTA results in optical flow estimation. Closest to our approach, GCNet  constructs hierarchical 4D feature volumes and processes them from coarse to fine using 3D convolutions, but we differ in our successful application of coarse-to-fine principles to high-resolution inputs and anytime, on-demand processing.
In this section, we describe the key ingredients of our approach: 1) an efficient hierarchical stereo matching architecture, 2) a set of novel asymmetric augmentation techniques, and 3) a high-resolution synthetic dataset for training. We also introduce a high-resolution stereo benchmark for real-world autonomous driving.
3.1 Hierarchical Stereo Matching (HSM) network
Our core idea of designing the hierarchical coarse-to-fine network is to first aggressively downsample high res images through the network while extracting multi-scale features, and then use potential correspondences to gradually build up a pyramid of cost volumes that increase in resolution. We provide precise layer and filter dimensions in the supplement.
Design principles: We find that coarse-to-fine design principles are crucial, specifically making use of 1) spatial pyramid pooling (SPP) , which allows features to dramatically increase in receptive field. Without this, features tend to have too small a receptive field compared to the rest of the high-res image. The original implementation in SPP 
upsampled pyramid features back to the original resolution. To reduce memory, we keep pooled features in their native coarse resolution; 2) 3D convolutions that are strided in the disparity dimension, allowing us to process high-res cost volumes efficiently; 3) multi-scale loss functions. The network architecture is shown in Figure2.
Feature pyramid encoder: We use a feature pyramid encoder to extract descriptors for coarse-to-fine matching. To efficiently extract features with different levels of details while maintaining the coarse-scale information, we adopt an encoder-decoder architecture with skip connections. Our feature encoder consists of custom resnet backbone with 4 residual blocks, followed by 4 SPP layers (again, to increase the receptive fields with limited computation and memory).
Feature volumes: We obtain such features for both left and right image, and then construct a D feature volume [5, 12, 13] by considering differences between pairs of potentially-matching descriptors along horizontal scanlines. We construct a pyramid of volumes, each with increasing spatial resolution and increasing disparity resolution. While cost volumes are traditionally D (height H by width W by disparity D), our feature volume includes a th dimension representing the number of feature channels C, which increases for later layers in the encoder.
Feature volume decoder: Figure 4 visualizes the decoding
, or filtering, of each feature volume. Let us first define a conv3D “block” as two 3D convolutions with a residual connection. 1) The feature volume is filtered byconv3D blocks. 2) As was the case for feature extraction, we then apply Volumetric Pyramid Pooling (our extension of SPP to feature volumes) to generate features that capture sufficient global context for high-res inputs. 3a) The output is trilinearly-upsampled to a higher spatial (and disparity) resolution so that it can be fused with the next 4D feature volume in the pyramid. 3b) To report on-demand disparities computed from the current scale, the output is processed with another conv3D block to generate a 3D output cost volume. This cost volume can directly report disparities before subsequent feature volumes downstream in the pyramid are ever computed.
Multi-scale loss: We train the network to make predictions at different scales in the training phase, which allows for on-demand disparity output at any pyramid level, and also serves to regularize the overall network:
where losses are scaled to account for the increased disparity resolution at each pyramid level. represents the loss on the finest level, and is the loss on the most coarse level. A natural loss is a softmax distribution over candidate disparities at the current pyramid level. We found that expected disparity, as in GCNet , worked better.
3.2 Stereo data augmentation
In order to train our network, we find it crucial to make use of high-res training data and specific strategies for data augmentation. We discuss both below. Most conventional stereo systems make several assumptions on the target and reference view image pairs, including 1) both images are under the same imaging condition, 2) cameras are perfectly calibrated and 3) there is no occlusion and each pixel can find a match. However, these assumptions do not always hold in real-world scenarios. We propose asymmetric augmentation techniques to tackle these issues accordingly for our learning-based approach.
y-disparity augmentation: Most stereo systems assume that cameras are perfectly calibrated and correspondences lie on the same horizontal scan-line. However, it is difficult to perfectly calibrate a high-res image pair, particularly during large temperature changes and vibrations . Such errors result in ground-truth disparity matches that have a component (e.g., match to a different horizontal scanline). Hierarchical matching partially mitigates this issue by biasing matching at the coarse scale, where calibration error is not as severe. Another approach is to force the network to learn robustness to such errors in training time. Notice that errors in camera calibration can be represented by a homography , resulting in where is the image coordinate. To mimic the real-world calibration error, we warp the target view image according to the calibration error matrix. To further constrain the space of the warped images, we limit to be a rigid 2D transformation.
Asymmetric chromatic augmentation: It is inevitable that stereo cameras may under different lighting and exposure conditions, for example, one camera is under the shadow. Therefore making algorithms robust to such imaging asymmetry is critical out of safety concern. We achieve this goal by applying different chromatic augmentation to both reference and target images, in the hope that our feature encoding network may learn a representation robust to such imaging variants.
Asymmetric masking: Most stereo matching algorithms assume that correspondences always exist in the target view. However, this assumption does not hold when occlusion occurs or correspondences are difficult to find. On the other hand, monocular cues, such as shape and continuity, as well as contextual information are found helpful in estimating the disparity. To force the model to rely more on contextual cues, we apply asymmetric masking, which randomly replaces a rectangular region in the target view with mean RGB values of the whole image .
3.3 High-resolution datasets
End-to-end high-resolution stereo matching requires high-resolution datasets to make it effective. However, as shown in Table 1, there exist very few such datasets for both training and testing purpose. Middlebury-v3  is the only publicly available dataset for high-resolution stereo matching. However, it contains very few samples and there are no outdoor/driving scenes. To bridge this gap, we aggregate two datasets for high-resolution stereo matching on driving scenes. Synthetic HR-VS is collected for training high-resolution stereo models, while the high-res real stereo (HR-RS) dataset is collected to benchmark high-resolution stereo matching methods under real-world driving scenes.
High-res virtual stereo (HR-VS) dataset:
HR-VS is collected using the open-sourced Carla simulator. Under weather conditions while maneuvering in Town01, we collected pairs of training data at resolution. Camera baseline and focal length are set as m and px respectively. Pixels with a depth greater than m or a disparity greater than px are removed to mimic the disparity distribution of real-world driving scenarios, resulting in a disparity range of px and depth range of m. Sample images and ground-truth can be found in the supplements.
High-res real stereo (HR-RS) benchmark: HR-RS includes pairs of images and disparity ground-truths collected using high-resolution stereo cameras and LiDAR while driving in the urban scenes. Images are rectified and cropped to . LiDAR point clouds are projected to image planes and converted to disparities, resulting in a range of px. We also manually removed point clouds on dynamic objects to reduce camera-LiDAR registration error.
Benchmark protocol: Since we are motivated by autonomous depth sensing, it is crucial to know the ideal sensing range under different driving speeds. Under the assumption of dry road condition and maximum deceleration of a car as shown in the supplements, we calculate the safe stopping distance for speed mph as m respectively. To ensure the safety of the driver and passengers, we are interested in making correct predictions within these stopping distances, i.e., when an object enters the stopping distance, we have to correctly perceive it. This gives us three sets of metrics with regard to driving speeds, which we refer to as short-range (0-25m), middle-range (25-60m) and long-range (60-115m) metrics. For each safe distance range, we use the same set of metrics in pixel space following .
In this section, we evaluate our hierarchical stereo matching (HSM) network on public benchmarks including high-res Middlebury-v3 and low-res KITTI, as well as our proposed high-resolution benchmark, i.e., HR-RS.
|HSM-F3 (Ours)||0.51 (0.61)||3.44||13.4||9.68||16.5||31.2||63.8||17.6||4.26|
|iResNet_ROB ||0.34 (0.42)||6.56||18.1||22.1||31.7||45.9||87.5||36.2||15.1|
Datasets: We used publicly available datasets, including Middlebury-v3, KITTI-15, ETH3D and Sceneflow [20, 21, 26, 28], as well as HR-VS dataset for training. Middlebury-v3 contains high-resolution training image pairs where each has variants with imperfect calibration, different exposures, and different lighting conditions, resulting in pairs in total. KITTI-15 contains low-resolution pairs and ETH3D contains low-resolution pairs with sparsely labeled ground-truth. Sceneflow contains around k synthetic image pairs with dense disparity labels. And HR-VS contains training pairs.
We implemented the HSM network using Pytorch. We train the model using Adam optimizer with a batch size of 24 on a machine withTesla V100 GPUs, while setting the initial learning rate to and betas to . We train for epochs and then shrink the learning rate by .
During training, we augment Middlebury, KITTI-15, ETH3D [20, 21, 26] and HR-VS to the same size as Sceneflow, resulting in around 170k training samples. We perform both symmetric and asymmetric augmentations on the fly. Asymmetric chromatic augmentations include randomly applying different brightness (), gamma () and contrast () to target and inference images. We apply y-disparity augmentation by uniformly sample a delta y-translation (px) and rotation() at a chance of . We also apply asymmetric masking at a chance of by uniformly sample the width (px) and height (px) of the mask and randomly placing it on the image. Symmetric augmentations include scaling ( for low-res images and for high-res images) and randomly cropping to fix-sized patches. We set the number of disparities for searching as px.
At test time, we make predictions at the last scales from coarse to fine. With full-res input, these predictions are referred to as “Our-F1”, “Our-F2” and “Our-F3” respectively, where “F” indicates the input resolution and the following number indicates the stage. Similarly, with half resolution input we have “Our-H2” and “Our-H3”; for quarter resolution inputs, we have “Our-Q3”. When testing on Middlebury-v3 images, we set the disparity search range according to maximum disparities in the calibration files, while for 1.8-times upscaled KITTI-15 test images, we set disparity search range as 384. For HR-RS images, we set disparity search range as 512.
Metrics: Different metrics are used for different benchmarks. On Middlebury-v3 , we divide official metrics into groups: 1) bad-4.0 (percentage of “bad” pixels whose error is great than 4.0), bad-2.0 and bad-1.0, which tolerates the small deviations such as quantization error. 2) avgerr (average absolute error in pixels) and rms (root-mean-square disparity error in pixels), which also takes into account the subpixel accuracy. 3) A99 (
error quantile in pixels), A95 and A90, which ignores large deviations while measuring the accuracy. On KITTI-15
, we use official metrics D1-all, D1-bg and D1-fg, which measure the percentage of outliers for all pixels, background pixels and foreground pixels respectively. While on HR-RS, we separate pixels to different depth ranges and use the same set of metrics as Middlebury-v3.
4.2 Benchmark performance
High-res Middlebury-v3: Because we could not submit multiple on-demand outputs to the online test server for Middlebury-v3, we evaluate only our full-res, full-pipeline model HSM-F3. We compare against two groups of published methods. The first group of methods includes those running under 1s/image, which we refer to as the “fast” group, and the second group of methods includes published SOTA that run slower. To compare with iResNet , we also add the additional images as they did in training time, and in test time, we take the full resolution input images as input.
We first present qualitative results, including comparisons with SOTA methods in Fig. 5 and reconstruction of Middlebury test scene “Plants” in Fig. 3. The quantitative comparisons to SOTA are shown in Table 2. Compared to MC-CNN-acrt on all-pixels, we reduced avgerr by , rms by and bad-4.0 by while running times faster. Compared to CBMV_ROB, we reduced avgerr by , rms by and bad-4.0 by while running times faster. We run times slower than iResNet_ROB but reduced avgerr by , rms by and bad-4.0 by . Notice that in submission time, we used Tesla V100 on Amazon AWS, which gives us 510ms/image on average while iResNet used Titan Xp. To be fair, we measured running time for HSM-F3 and “iResNet” on the same Titan X Pascal GPU, as shown in parenthesis. We are slower, but much more accurate.
Low-res KITTI: Though our focus is not on low-res, we still evaluate on KITTI-15 and partition the SOTA algorithms into two groups: those that run under ms and fits real-time needs111Since many autonomous robots employ sensors synced at 10fps, the deployment-ready version of a 200ms model is likely to fit real-time needs., while the other group is slower but more accurate. During training time, we excluded the ETH3D and Sceneflow datasets and finetuned with Middlebury-v3, KITTI-12 and KITTI-15. At test time, we operate on image pairs up-sampled by a factor of . We first show qualitative results in Figure 6. Then we refer to Table 3 for quantitative results. We rank st among all published methods while running times faster than “EdgeStereo”, which ranks nd among the published methods.
4.3 Results on HR-RS
Then we evaluate on HR-RS and compare against a subset of the previous arts under the protocol discussed in method section. ELAS  is taken from the Robust Vision Challenge official package, iResNet  is taken from their Github repository, and we implemented two-pass SGBM2  using OpenCV (with SAD window size = 3, truncation value for pre-filter = 63, p1 = 216, p2 = 864, uniqueness ratio = 10, speckle window size = 100, speckle range = 32). The results from SGBM2 is also post-processed using weighted least square filter with default parameters. “HSM-F2” means the model operates on full resolution images and make predictions at the second stage (2nd last scale). Results are shown in Table 4.
Anytime on-demand: Cutting off HSM-F at the second stage (HSM-F2), bad-4.0 on all pixels increases by while being x faster, which is still more accurate than ELAS, SGBM and iResNet. Same as iResNet-H, HSM-H3 uses half resolution image as input, but runs x faster and produces 29.5% lower bad-4.0. Halting earlier (HSM-H2) increases error by 47.8% but is 0.76x faster, which is still more accurate than ELAS and SGM.
Long-range sensing: We analyze our methods for different distance ranges. Halting HSM-F earlier at the second stage (HSM-F2) only increases bad-4.0 on short-range pixels by 5.1%, which suggests that high-res inputs might not help. However, on long-range pixels it increases bad-4.0 by 14.8%. Interestingly, halting HSM-F earlier (HSM-F2) still produces more accurate long-range predictions than HSM-H3 (17.1% versus 19.7%). This is possibly because the features pyramid already encodes detailed information from high-res inputs, which helps to predict accurate long-range disparities.
We perform a detailed ablation study to reveal the contribution of each individual component of our method. We follow the same training protocol as described in the experiment setup, but initialize the encoder weights using the same fine-tuned model. We train with a batch size of for k iterations. The learning rate is down-scaled by for the last k iterations. We then test on half-resolution additional Middlebury images except for ”Shopvac”, resulting in 24 test images in total. Quantitative results are shown in Table 5.
Feature volume fusion: When aggregating information across pyramid scales, we made the design choice to fuse coarse-scale 4D feature volumes instead of 3D cost volume. Our intuition was that “feature-volume fusion” tolerates incorrect coarse predictions since the final output does not directly depend on initial predictions. Also as shown in Table 5, we found that replacing “feature-volume fusion” with “cost-volume fusion” results in more bad pixels with an error greater than px.
High-res synthetic data: After removing HR-VS dataset when training the network, the bad-1.0 metric increases by , and the bad-2.0 metric increases by , which shows the importance of adding synthetic high-res data when training a high-res stereo network.
y-disparity augmentation: y-disparity augmentation is an effective way of forcing the network to learn features robust to camera calibration error. As shown in Table 5, removing y-disparity augmentation increase bad-1.0 by .
Multi-scale loss: Multi-scale loss regularizes the network by forcing it to learn multiple prediction tasks, while also helps gradients propagate through multiple scales. We found that removing Multi-scale loss increase bad-2.0 by .
Asymmetric masking: Asymmetric masking is a powerful tool that encourages the network to learn to deal with occlusions. After removing “asymmetric masking” data augmentation, bad-2.0 error increases by 4.9%.
Volumentric pyramid pooling Volumetric pyramid pooling effectively enlarges the receptive field of the decoder network, which is important when the input resolution is high. Removing VPP results in an increase of bad-2.0 error by 3.2%.
Asymmetric chromatic augmentation:
We found removing asymmetric chromatic augmentation does not harm performance on Middlebury additional images. This is probably because we introduce extra noise into training pairs, which harms our predictions on normal images (with the same exposure/lighting). However, we found this technique is helpful in making the network robust to asymmetric imaging conditions.
4.5 Robustness to adversarial conditions
Robustness to camera occlusion In real-world autonomous driving scenarios, it may occur that one of the stereo cameras is blurred by the raindrops or occluded by the snowflakes, which introduces challenges to stereo matching algorithms. To study the effect of occlusion to our model, we place a rectangular gray patch at the target image center while keeping the reference view unchanged. Predictions from our model are shown in Fig. 7 and Fig. 8. We find that with an increased amount of occluded pixels, the average error grows slowly at the beginning (0.1% from 0 to with 5625 occluded pixels), and faster towards the end. Our model is capable of handling small camera occlusions, possibly by inferring disparities based on monocular and context cues.
Resilience to calibration error We also compare our HSM-H model against ELAS-H  in terms of robustness to calibration error as shown in Tab. 6. We find our model is more robust to calibration errors: when the camera becomes imperfectly calibrated, the average pixel error rate of HSM-H only increases 5.9%, while that of ELAS-H increases by 29.6%. The robustness of our model is due to the coarse-to-fine matching process where the calibration error is suppressed in the coarse scale. Our y-disparity augmentation is also helpful in that it forces the network to learn features robust to y-disparities.
We then characterize the sensitivity of our model to calibration errors as follows: we select images with perfect calibration (12 in total) and 1) rotate the target view images around center for degrees, as well as 2) translate along y-axis for pixels, while keeping the reference view unchanged. The resulting avgerr to is shown in Fig. 9. We find that under a small rotation of 0.05 degrees (resulting 1.07 px y-disparity around the image border), our model’s error rate only increases by 5.3%, while under a large rotation of 0.4 degrees (resulting 8.6 px y-disparity around the image border), our model’s error increases by 105%, while still better than the results of ELAS-H on perfectly calibrated images (8.0 vs 8.3). Also under a small translation of 1px, our model’s error rate almost does not increase (only rise by 1.0%), while under a large translation of 8 px, our model’s error increases by 51.5%.
Robustness to change of exposure We further compare our HSM-H model to ELAS-H in terms of robustness to exposure change and robustness to lighting changes as shown in Tab. 7, where we found our model to be more robust: when exposure changes, the average error of HSM-H only increases 8.2%, while that of ELAS-H increases by 44.2%. The robustness of our model also shows the effectiveness of asymmetric chromatic augmentation, which forces the model to learn features robust to chromatic changes.
Robustness to change of lighting We finally test HSM-H and ELAS-H in terms of robustness to lighting changes, and results are shown in Tab. 8. We found that when lighting changes, the average error of both HSM-H and ELAS-H increase a lot. We also include a failure case for both our model and ELAS as in Fig 10.
With the help of hierarchical designs, high-resolution synthetic dataset and asymmetric augmentation techniques, our model achieves SOTA performance on Middlebury-v3 and KITTI-15 while running significantly faster than prior arts. We are also able to perform on-demand disparity estimation at different scales, making possible accurate depth prediction of close-by objects in real-time.
Acknowledgements: This work was supported by the CMU Argo AI Center for Autonomous Vehicle Research.
-  (1989) A computational framework and an algorithm for the measurement of visual motion. International Journal of Computer Vision 2 (3), pp. 283–310. Cited by: §1.
-  (2018) CBMV: a coalesced bidirectional matching volume for disparity estimation. In CVPR, Cited by: Figure 5, Table 2.
-  (2011) Large displacement optical flow: descriptor matching in variational motion estimation. IEEE transactions on pattern analysis and machine intelligence 33 (3), pp. 500–513. Cited by: §1.
-  (2012) A naturalistic open source movie for optical flow evaluation. In ECCV, Cited by: Table 1.
-  (2018) Pyramid stereo matching network. In CVPR, Cited by: §1, §2, §3.1, Table 2, Table 3.
-  (2017) CARLA: An open urban driving simulator. In CoRL, Cited by: §3.3.
-  (2010) Efficient large-scale stereo matching. In ACCV, Cited by: §2, §4.3, §4.5, Table 2, Table 4, Table 6, Table 7, Table 8.
-  (2009) Stereo matching in the presence of sub-pixel calibration errors. In CVPR, Cited by: §3.2.
-  (2008) Stereo processing by semiglobal matching and mutual information. IEEE Transactions on pattern analysis and machine intelligence 30 (2), pp. 328–341. Cited by: §2, §4.3, Table 2, Table 4.
-  (2013) Fast cost-volume filtering for visual correspondence and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence 35 (2), pp. 504–511. Cited by: §2.
-  (2018) Occlusions, motion and depth boundaries with a generic network for disparity, optical flow or scene flow estimation. In ECCV, Cited by: §1, Table 2, Table 3.
-  (2017) End-to-end learning of geometry and context for deep stereo regression. In ICCV, Cited by: §2, §2, §3.1, §3.1, Table 3.
-  (2018) Stereonet: guided hierarchical refinement for real-time edge-aware depth prediction. In ECCV, Cited by: §2, §3.1, Table 3.
-  (2001) Computing visual correspondence with occlusions using graph cuts. In ICCV, Cited by: §2.
-  (2017) High-resolution stereo matching based on sampled photoconsistency computation. In BMVC, Cited by: §2, Table 2.
-  (2018) Learning for disparity estimation through feature constancy. In CVPR, Cited by: §1, §2, §4.2, §4.3, Table 2, Table 3, Table 4.
-  (2015) Fully convolutional networks for semantic segmentation. In CVPR, Cited by: §2.
-  (1981) An iterative image registration technique with an application to stereo vision. Cited by: §1.
Efficient deep learning for stereo matching. In CVPR, Cited by: §2.
-  (2016) A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In CVPR, Cited by: §2, §2, Table 1, §4.1, §4.1, Table 3.
-  (2015) Object scene flow for autonomous vehicles. In CVPR, Cited by: §1, Table 1, §4.1, §4.1, §4.1.
-  (1985) Stereo by intra-and inter-scanline search using dynamic programming. IEEE Transactions on pattern analysis and machine intelligence (2), pp. 139–154. Cited by: §2.
Cascade residual learning: a two-stage convolutional neural network for stereo matching. In ICCV, Cited by: §2, §2, Table 3.
-  (2014) Know your limits: accuracy of long range stereoscopic object measurements in practice. In ECCV, Cited by: §1.
-  (2015) U-net: convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pp. 234–241. Cited by: §2.
High-resolution stereo datasets with subpixel-accurate ground truth.
German Conference on Pattern Recognition, Cited by: §1, §3.3, §3.3, Table 1, §4.1, §4.1, §4.1.
-  (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision 47 (1-3), pp. 7–42. Cited by: §2.
-  (2017) A multi-view stereo benchmark with high-resolution images and multi-camera videos. In CVPR, Cited by: Table 1, §4.1.
-  (2017) Hide-and-seek: forcing a network to be meticulous for weakly-supervised object and action localization. In ICCV, Cited by: §3.2.
-  (2014) Efficient high-resolution stereo matching using local plane sweeps. In CVPR, Cited by: §2, Table 2.
-  (2018) EdgeStereo: a context integrated residual pyramid network for stereo matching. In ACCV, Cited by: Table 3.
-  (2008) On benchmarking camera calibration and multi-view stereo for high resolution imagery. In CVPR, Cited by: §1, §2.
-  (2018) PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. In CVPR, Cited by: §2.
-  (2018) Continuous 3d label stereo matching using local expansion moves. IEEE transactions on pattern analysis and machine intelligence 40 (11), pp. 2725–2739. Cited by: Figure 5, Table 2.
-  (2008) A fast local descriptor for dense matching. In CVPR, Cited by: §2.
-  (2010) Daisy: an efficient dense descriptor applied to wide-baseline stereo. IEEE transactions on pattern analysis and machine intelligence 32 (5), pp. 815–830. Cited by: §2.
-  (2018) Practical deep stereo (pds): toward applications-friendly deep stereo matching. In NeurIPS, Cited by: Table 3.
-  (2018) SegStereo: exploiting semantic information for disparity estimation. In ECCV, Cited by: §1, Table 3.
-  (2006) Real-time global stereo matching using hierarchical belief propagation.. In BMVC, Cited by: §1.
Stereo matching by training a convolutional neural network to compare image patches.
Journal of Machine Learning Research17 (1-32), pp. 2. Cited by: §2, Table 2.
-  (2017) Pyramid scene parsing network. In CVPR, Cited by: §3.1.