Deep Learning Models for Architectural Façade Detection in Spherical Images

Abstract

Maintaining a public register by extracting urban objects from photos taken in the city is one of the most important tasks for municipal services. It is of great importance in the field of protection and shaping of the cultural landscape, protection of monuments, and registration of the urban tissue development. The current state of the art shows that deep learning models (DL models) can cope with the problem of extracting urban objects with the same or better performance than non-DL models, and can process video and photos automatically. This paper compares the three main DL models for facade instance detection and facade segmentation: Mask R-CNN, YOLACT, and Mask-Scoring R-CNN. The training and validation datasets used for transfer learning were created on the basis of spherical photos taken in an artificially generated virtual city. The test dataset, on the other hand, included spherical façade photos taken in a real city. The comparative analysis of the DL models was performed using parametric and nonparametric statistical tests for pairwise and multiple comparisons.