Energy-based multi-view piecewise planar stereo

Abstract

The piecewise planar model (PPM) is an effective means of approximating a complex scene by using planar patches to give a complete interpretation of the spatial points reconstructed from projected 2D images. The traditional piecewise planar stereo methods suffer from either a very restricted number of directions for plane detection or heavy reliance on the segmentation accuracy of superpixels. To address these issues, we propose a new multi-view piecewise planar stereo method in this paper. Our method formulates the problem of complete scene reconstruction as a multi-level energy minimization problem. To detect planes along principal directions, a novel energy formulation with pair-wise potentials is used to assign an optimal plane for each superpixel in an iterative manner, where reliable scene priors and geometric constraints are incorporated to enhance the modeling efficacy and inference efficiency. To detect non-principal-direction planes, we adopt a multi-direction plane sweeping with a restricted search space method to generate reliable candidate planes. To handle the multi-surface straddling problem of a single superpixel, a superpixel sub-segmenting scheme is proposed and a robust P ⁿ Potts model-like higher-order potential is introduced to refine the resulting depth map. Our method is a natural integration of pixel- and superpixel-level multi-view stereos under a unified energy minimization framework. Experimental results for standard data sets and our own data sets show that our proposed method can satisfactorily handle many challenging factors (e.g., slanted surfaces and poorly textured regions) and can obtain accurate piecewise planar depth maps.

创新点

对于基于图像的三维场景重建,由于光照变化、透视畸变、弱纹理等干扰因素的存在,传统像素级与区域级的重建算法通常难以获得可靠的结果。为了解决此问题,本文提出一种新颖的基于能量的场景分段平面重建算法。根据场景分段平面假设,本文算法在MRF(Markov Random Field)能量最小化框架下将场景完整结构推断问题转换为沿场景主方向与非主方向的平面标记以及平面级的场景结构优化等问题进行求解,由于候选平面集与融合灰度一致性度量、空间几何与可见性约束、空间平面先验的能量函数的高可靠性,因而可以快速获取完整、准确的场景模型。实验结果表明,本文算法不但可以有效地解决场景中弱纹理、倾斜表面等区域的重建问题,而且可以克服传统相关算法依赖特定场景模型(如Manhattan 场景模型)、易受图像过分割精度的影响等缺点,整体上具有较高的可靠性与效率。