Abstract
In this paper we present a hybrid method for detecting windows on high-resolution rectified images of building facades combining deep learning with traditional geometric processing. As initial step we use a deep learning object detection method. As we observed that in most cases the detector outputs a larger object than the ground truth. We employ geometric processing based on image gradients to precisely delineate the window edges. For the evaluation of the algorithm we have created a high resolution dataset with more than 2000 annotated windows. The obtained results show that the detector’s bounding box differs from ground truth mostly by less than six pixels. The Intersection over Union IoU of the objects is 96.9%. Geometric processing improves IoU by 1.7% leading to an IoU score of 98.6%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ali, H., Seifert, C., Jindal, N., Paletta, L., Paar, G.: Window detection in facades. In: International Conference on Image Analysis and Processing, pp. 837–842 (2007)
Hensel, S., Goebbels, S., Kada, M.: Facade reconstruction for textured LoD2 CityGML models based on deep learning and mixed integer linear programming. ISPRS Ann. Photogrammetry Remote Sens. Spat. Inf. Sci. 4 (2019)
Hu, H., Wang, L., Zhang, M., Ding, Y., Zhu, Q.: Fast and regularized reconstruction of building facades from street-view images using binary integer programming. arXiv preprint arXiv:2002.08549 (2020)
Lazebnik, S.: Ultralytics. YOLOv5. https://github.com/ultralytics/yolov5
Lee, S.C., Nevatia, R.: Extraction and integration of window in a 3D building model from ground view images. In: Computer Vision and Pattern Recognition, vol. 2, p. II (2004)
Ley, A., Hellwich, O.: Depth map based facade abstraction from noisy multi-view stereo point clouds. In: Rosenhahn, B., Andres, B. (eds.) GCPR 2016. LNCS, vol. 9796, pp. 155–165. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-45886-1_13
Li, C.K., Zhang, H.X., Liu, J.X., Zhang, Y.Q., Zou, S.C., Fang, Y.T.: Window detection in facades using heatmap fusion. J. Comput. Sci. Technol. 35(4), 900–912 (2020)
Liu, H., Zhang, J., Zhu, J., Hoi, S.C.H.: DeepFacade: a deep learning approach to facade parsing. In: International Joint Conference on Artificial Intelligence, pp. 2301–2307 (2017)
Ma, W., Ma, W.: Deep window detection in street scenes. KSII Trans. Internet Inf. Syst. (TIIS) 14(2), 855–870 (2020)
Martinovic, A., Van Gool, L.: Bayesian grammar learning for inverse procedural modeling. In: Computer Vision and Pattern Recognition, pp. 201–208 (2013)
Rahmani, K., Huang, H., Mayer, H.: Facade segmentation with a structured random forest. ISPRS Ann. Photogrammetry Remote Sens. Spat. Inf. Sci. 4, 175–181 (2017)
Rahmani, K., Mayer, H.: High quality facade segmentation based on structured random forest, region proposal network and rectangular fitting. ISPRS Ann. Photogrammetry Remote Sens. Spat. Inf. Sci. 4(2) (2018)
Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement. arXiv preprint arXiv:1804.02767 (2018)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)
Reznik, S., Mayer, H.: Implicit shape models, self-diagnosis, and model selection for 3D façade interpretation. Photogrammetrie Fernerkundung Geoinformation, pp. 187–196 (2008)
Riemenschneider, H., Bódis-Szomorú, A., Weissenberg, J., Van Gool, L.: Learning where to classify in multi-view semantic segmentation. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 516–532. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_34
Riemenschneider, H., et al.: Irregular lattices for complex shape grammar facade parsing. In: Computer Vision and Pattern Recognition, pp. 1640–1647 (2012)
Russell, B.C., Torralba, A., Murphy, K.P., Freeman, W.T.: LabelMe: a database and web-based tool for image annotation. Int. J. Comput. Vis. 77(1–3), 157–173 (2008)
Schmitz, M., Mayer, H.: A convolutional network for semantic facade segmentation and interpretation. Int. Arch. Photogrammetry Remote Sens. Spat. Inf. Sci. XLI(133), 709–715 (2016)
Teboul, O., Kokkinos, I., Simon, L., Koutsourakis, P., Paragios, N.: Shape grammar parsing via reinforcement learning. In: Computer Vision and Pattern Recognition, pp. 2273–2280 (2011)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Rahmani, K., Mayer, H. (2022). A Hybrid Method for Window Detection on High Resolution Facade Images. In: Bennour, A., Ensari, T., Kessentini, Y., Eom, S. (eds) Intelligent Systems and Pattern Recognition. ISPR 2022. Communications in Computer and Information Science, vol 1589. Springer, Cham. https://doi.org/10.1007/978-3-031-08277-1_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-08277-1_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-08276-4
Online ISBN: 978-3-031-08277-1
eBook Packages: Computer ScienceComputer Science (R0)