Abstract
Computer vision techniques are increasingly used to develop efficient and automatic methods that provide alternative data sources. Micro-level information on buildings and physical infrastructure is very important to social, economic, and environmental statistical programs. In this paper, we present how multiple convolutional neural networks (CNNs) are finetuned to classify residential buildings into five different types (e.g., single-detached, semi-detached, condominium apartment, etc.) from their street-view images. A framework for collection and automatic preliminary labeling of street-view images is developed and presented. Furthermore, a big dataset of street-view images of residential buildings has been compiled and labeled to be used for training. Multiple state-of-the-art CNNs are finetuned to accomplish the classification task. The trained models provide a proof of concept and show that CNNs can be used to classify residential buildings using their street-view imagery. The performance of the trained CNNs is measured and presented. This approach can be used to augment the information available on openly accessible databases, such as the Open Database of Buildings.
Similar content being viewed by others
Data availability
The data in this work cannot be made publicly available due to the terms of use.
References
Giri, C.: Remote sensing of land use and land cover. CRC Press (2016)
Li, X., Zhang, C., Li, W.: Building block level urban land-use information retrieval based on google street view images. GIScience Remote Sens 54, 819–835 (2017). https://doi.org/10.1080/15481603.2017.1338389
Gebru, T., Krause, J., Wang, Y., et al.: Using deep learning and google street view to estimate the demographic makeup of neighborhoods across the United States. In: Proceedings of the National Academy of Sciences of the United States of America. National Academy of Sciences, pp 13108–13113 (2017)
Kang, J., Körner, M., Wang, Y., et al.: Building instance classification using street view images. ISPRS J. Photogramm. Remote Sens. 145, 44–59 (2018). https://doi.org/10.1016/j.isprsjprs.2018.02.006
Google. Street View Static API. https://developers.google.com/maps/documentation/streetview/intro. (2020). Accessed 1 Jul 2020
Mapillary. Mapillary. https://www.mapillary.com/. (2014). Accessed 16 Jul 2020
Grab Holdings. Open Street Cam. https://openstreetcam.org/map/@45.37744755572422,-75.65142697038783,18z. (2009). Accessed 1 Sep 2020
Raja, M.N.A., Abdoun, T., El-Sekelly, W.: Smart prediction of liquefaction-induced lateral spreading. J. Rock Mech. Geotech. Eng. (2023). https://doi.org/10.1016/J.JRMGE.2023.05.017
Teizer, J., Vela, P.A.: Personnel tracking on construction sites using video cameras. Adv. Eng. Inf. 23, 452–462 (2009). https://doi.org/10.1016/J.AEI.2009.06.011
OpenStreetMap. OpenStreetMap. https://www.openstreetmap.org/#map=4/30.56/-64.16. (2004). Accessed 16 Jul 2020
Voulodimos, A., Doulamis, N., Doulamis, A., Protopapadakis, E.: Deep learning for computer vision: a brief review. Comput. Intell. Neurosci. 2018, 1–13 (2018). https://doi.org/10.1155/2018/7068349
Biljecki, F., Ito, K.: Street view imagery in urban analytics and GIS: a review. Landsc. Urban Plan. 215, 1–20 (2021). https://doi.org/10.1016/J.LANDURBPLAN.2021.104217
Smith, V., Malik, J., Culler, D.: Classification of sidewalks in street view images. In: 2013 International Green Computing Conference Proceedings, IGCC 2013. IEEE Computer Society, (2013) pp 1–6
Marianingsih, S., Utaminingrum, F.: Comparison of support vector machine classifier and Naïve bayes classifier on road surface type classification. In: 3rd Int Conf Sustain Inf Eng Technol SIET 2018—Proc 48–53. (2018). https://doi.org/10.1109/SIET.2018.8693113
Gonzalez, D., Rueda-Plata, D., Acevedo, A.B., et al.: Automatic detection of building typology using deep learning methods on street level images. Build. Environ. 177, 106805 (2020). https://doi.org/10.1016/J.BUILDENV.2020.106805
Li, Y., Chen, Y., Rajabifard, A., et al.: Estimating building age from google street view images using deep learning. In: 10th International Conference on Geographic Information Science. Schloss Dagstuhl—Leibniz-Zentrum fuer Informatik, pp 1–7. (2018).
Al-Habashna A (2021) Building height estimation using street-view images, deep-learning, contour processing, and geospatial data. In: Proceedings—2021 18th Conference on Robots and Vision, CRV 2021 103–110. https://doi.org/10.1109/CRV52889.2021.00022
OpenStreetMap. Overpass API - OpenStreetMap Wiki. https://wiki.openstreetmap.org/wiki/Overpass_API. (2022). Accessed 23 Feb 2022
Al-Habashna, A.: An open-source system for building-height estimation using street-view images, deep learning, and building footprints. Stat. Canada Artic Rep. (2020)
Microsoft. Visual Object Tagging Tool. https://github.com/Microsoft/VoTT. (2020). Accessed 23 Feb 2022
Quinn, J. et al.: Dive into deep learning: tools for engagement paperback, 1st ed. Corwin. (2019)
Russakovsky, O., Deng, J., Su, H., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, (2016). pp 770–778
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: The 3rd International Conference on Learning Representations. International Conference on Learning Representations, ICLR. (2015)
PyTorch. PyTorch. https://pytorch.org/. (2016). Accessed 21 Apr 2021
Acknowledgements
The authors would like to thank their colleagues Marian Radulescu, Herménégilde Nkurunziza, and Parvin Soleymani-Olyaei from CHSP, Statistics Canada for the valuable input and insightful discussions.
Funding
This project was funded by the Canadian Housing Statistics Program (CHSP) Division, Statistics Canada.
Author information
Authors and Affiliations
Contributions
RM and AA developed the methods in this work, carried out the experiments, discussed the results, and contributed to the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no competing interests to disclose.
Ethical approval
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Murdoch, R., Al-Habashna, A. Residential building type classification from street-view imagery with convolutional neural networks. SIViP 18, 1949–1958 (2024). https://doi.org/10.1007/s11760-023-02882-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-023-02882-8