A Noise-Driven Paradigm for Solving the Stereo Correspondence Problem

Abstract

The conventional technique for scene reconstruction from stereo image pairs searches for the best single surface fitting identified correspondences between the the two images. Constraints on surface continuity, smoothness, and visibility (occlusions) are incorporated into a ‘cost’ – usually an ad hoc linear combination of signal similarity criteria, with empirically selected coefficients. An unsatisfactory feature of this approach is that matching accuracy is very sensitive to correct choice of these coefficients. Also, few real scenes have only one surface, so that the single surface assumption contributes to matching errors.

We propose a noise-driven paradigm for stereo matching that does not couple the matching process with choice of surfaces by imposing constraints in the matching step. We call our strategy ‘Concurrent Stereo Matching’ because the first step involves a high degree of parallelism (making real-time implementations possible using configurable hardware): rather than search for ‘best’ matches, it first identifies all 3D volumes that match within a criteria based on noise in the image. Starting in the foreground, these volumes are then examined and surfaces are selected which exhibit high signal similarity in both images. Local constraints on continuity and visibility – rather than global ones – are used to select surfaces from the candidates identified in the first step.