Stereo Image Analysis of Non-Lambertian Surfaces

Abstract

Stereo image analysis is based on establishing correspondences between a pair of images by determining similarity measures for potentially corresponding image parts. Such similarity criteria are only strictly valid for surfaces with Lambertian (diffuse) reflectance characteristics. Specular reflections are viewpoint dependent and may thus cause large intensity differences at corresponding image points. In the presence of specular reflections, traditional stereo approaches are often unable to establish correspondences at all, or the inferred disparity values tend to be inaccurate, or the established correspondences do not belong to the same physical surface point. The stereo image analysis framework for non-Lambertian surfaces presented in this contribution combines geometric cues with photometric and polarimetric information into an iterative scheme that allows to establish stereo correspondences in accordance with the specular reflectance behaviour and at the same time to determine the surface gradient field based on the known photometric and polarimetric reflectance properties. The described approach yields a dense 3D reconstruction of the surface which is consistent with all observed geometric and photopolarimetric data. Initially, a sparse 3D point cloud of the surface is computed by traditional blockmatching stereo. Subsequently, a dense 3D profile of the surface is determined in the coordinate system of camera 1 based on the shape from photopolarimetric reflectance and depth technique. A synthetic image of the surface is rendered in the coordinate system of camera 2 using the illumination direction and reflectance properties of the surface material. Point correspondences between the rendered image and the observed image of camera 2 are established with the blockmatching technique. This procedure yields an increased number of 3D points of higher accuracy, compared to the initial 3D point cloud. The improved 3D point cloud is used to compute a refined dense 3D surface profile. These steps are iterated until convergence of the 3D reconstruction. An experimental evaluation of our method is provided for areas of several square centimetres of forged and cast iron objects with rough surfaces displaying both diffuse and significant specular reflectance components, where traditional stereo image analysis largely fails. A comparison to independently measured ground truth data reveals that the root-mean-square error of the 3D reconstruction results is typically of the order 30–100 μm at a lateral pixel resolution of 86 μm. For two example surfaces, the number of stereo correspondences established by the specular stereo algorithm is several orders of magnitude higher than the initial number of 3D points. For one example surface, the number of stereo correspondences decreases by a factor of about two, but the 3D point cloud obtained with the specular stereo method is less noisy, contains a negligible number of outliers, and shows significantly more surface detail than the initial 3D point cloud. For poorly known reflectance parameters we observe a graceful degradation of the accuracy of 3D reconstruction.