Multi-view Coherence for Outdoor Reflective Surfaces

Abstract

In this work, we address the deficiency that traditional multi-view matching algorithms cannot be applied to outdoor reflective surfaces. Since glass reflects different scenes under different viewpoints, which violates the multi-view coherence, it leads to the traditional 3D reconstruction algorithm to produce wrong estimation of camera poses, which eventually leads to the failure of reconstruction of outdoor buildings with glass surfaces. This paper proposes a pipeline that can improve the accuracy of reconstructing outdoor reflective surface. Firstly, we use a drone path planning algorithm that enables the dataset captured by the drone to have the maximum 3D reconstruction capability. Then, we propose a multi-view matching algorithm based on control points, which largely improves the accuracy and robustness of the current 3D reconstruction system. Finally, in order to restore the texture details of the 3D model, we replace the inaccurate texture mapping automatically generated by the modeling software with a new texture mapping through texture coordinate remapping. These three works together constitute a novel system that can reconstruct outdoor scenes with reflections in high quality. Finally, we apply our method to a variety of outdoor scenes with reflective surfaces, such as teaching buildings, libraries, and shopping malls, and we find that our method can produce the most excellent 3D reconstruction results.