Abstract
Floor plan analysis and vectorization are of practical importance in real estate and interior design fields. The analysis usually serves as a preliminary to the vectorization by extracting structural elements and room layouts. However, existing analysis methods mainly focus on the visual modality, which is insufficient for identifying rooms due to the lack of semantic clues about room types. On the other hand, standard floor plan images have rich textual annotations that provide semantic guidance of room layouts. Motivated by this fact, we propose a multimodal segmentation network (OCR)\(^2\) that exploits additional textual information for the analysis of floor plan images. Specifically, we extract texts that indicate the room layouts with optical character recognition (OCR) and fuse them with visual features by a cross-attention mechanism. Thereafter, we further optimize the state-of-the-art vectorization method in efficiency by (1) replacing the gradient-descent steps with the fast principle components analysis (PCA) to convert doors and windows, and (2) removing the unnecessary iterative steps when extracting room contours. Both quantitative and qualitative experiments validate the effectiveness and efficiency of our proposed method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ahmed, S., Liwicki, M., Weber, M., Dengel, A.: Improved automatic analysis of architectural floor plans. In: 2011 International Conference on Document Analysis and Recognition, pp. 864–869. IEEE (2011)
Ahmed, S., Liwicki, M., Weber, M., Dengel, A.: Automatic room detection and room labeling from architectural floor plans. In: 2012 10th IAPR International Workshop on Document Analysis Systems, pp. 339–343. IEEE (2012)
Ahmed, S., Weber, M., Liwicki, M., Dengel, A.: Text/graphics segmentation in architectural floor plans. In: 2011 International Conference on Document Analysis and Recognition, pp. 734–738. IEEE (2011)
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 833–851. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_49
Dodge, S., Xu, J., Stenger, B.: Parsing floor plan images. In: 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA), pp. 358–361. IEEE (2017)
Dosch, P., Tombre, K., Ah-Soon, C., Masini, G.: A complete system for the analysis of architectural drawings. Int. J. Doc. Anal. Recogn. 3(2), 102–116 (2000)
Douglas, D.H., Peucker, T.K.: Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartographica Int. J. Geographic Inf. Geovisual. 10(2), 112–122 (1973)
de las Heras, L.P., Ahmed, S., Liwicki, M., Valveny, E., Sánchez, G.: Statistical segmentation and structural recognition for floor plan interpretation. Int. J. Doc. Anal. Recogn. (IJDAR) 17(3), 221–237 (2014)
Liu, C., Wu, J., Kohli, P., Furukawa, Y.: Raster-to-vector: revisiting floorplan transformation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2195–2203 (2017)
Liu, S., Li, T., Chen, W., Li, H.: Soft rasterizer: a differentiable renderer for image-based 3d reasoning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7708–7717 (2019)
Lv, X., Zhao, S., Yu, X., Zhao, B.: Residential floor plan recognition and reconstruction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16717–16726 (2021)
Macé, S., Locteau, H., Valveny, E., Tabbone, S.: A system to detect rooms in architectural floor plan images. In: Proceedings of the 9th IAPR International Workshop on Document Analysis Systems, pp. 167–174 (2010)
Or, S.H., Wong, K.H., Yu, Y.K., Chang, M.M.V., Kong, H.: Highly automatic approach to architectural floorplan image understanding & model generation. Pattern Recogn. pp. 25–32 (2005)
Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32, pp. 8024–8035. Curran Associates, Inc. (2019), https://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf
Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Surikov, I.Y., Nakhatovich, M.A., Belyaev, S.Y., Savchuk, D.A.: Floor plan recognition and vectorization using combination UNet, Faster-RCNN, statistical component analysis and Ramer-Douglas-Peucker. In: Chaubey, N., Parikh, S., Amin, K. (eds.) COMS2 2020. CCIS, vol. 1235, pp. 16–28. Springer, Singapore (2020). https://doi.org/10.1007/978-981-15-6648-6_2
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Yuan, Y., Chen, X., Wang, J.: Object-contextual representations for semantic segmentation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12351, pp. 173–190. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58539-6_11
Zeng, Z., Li, X., Yu, Y.K., Fu, C.W.: Deep floor plan recognition using a multi-task network with room-boundary-guided attention. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9096–9104 (2019)
Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2881–2890 (2017)
Ziran, Z., Marinai, S.: Object detection in floor plan images. In: Pancioni, L., Schwenker, F., Trentin, E. (eds.) ANNPR 2018. LNCS (LNAI), vol. 11081, pp. 383–394. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99978-4_30
Acknowledgements
This work is supported by the National Natural Science Foundation of China (No. U1903214, 61876135, 61862015). The numerical calculations in this paper have been done on the supercomputing system in the Supercomputing Center of Wuhan University.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wen, T., Liang, C., Fu, YM., Xiao, CX., Xiang, HM. (2023). Floor Plan Analysis and Vectorization with Multimodal Information. In: Dang-Nguyen, DT., et al. MultiMedia Modeling. MMM 2023. Lecture Notes in Computer Science, vol 13833. Springer, Cham. https://doi.org/10.1007/978-3-031-27077-2_22
Download citation
DOI: https://doi.org/10.1007/978-3-031-27077-2_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-27076-5
Online ISBN: 978-3-031-27077-2
eBook Packages: Computer ScienceComputer Science (R0)