Skip to main content

Advertisement

SpringerLink
Log in

Research on text detection on building surfaces in smart cities based on deep learning

  • Data analytics and machine learning
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

In recent years, with the construction and development of smart cities, text recognition in building images can not only achieve geolocation but also provide guiding significance for GIS mapping and automatic updating. Since buildings have different orientations, angles and shapes, it is difficult to recognize textual features in images. With the wide application of convolutional neural networks and recurrent neural networks in image processing, this paper proposes a BFPN-RCNN algorithm for detecting and recognizing curved text in architectural images. A comparison with other image detection algorithms on different datasets proves that the algorithm can effectively identify curved text at different angles in natural scene images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data availability

Enquiries about data availability should be directed to the authors.

References

  • Batty M (2013) Big data, smart cities and city planning. Dialogues Hum Geogr 3(3):274–279

    Article  Google Scholar 

  • Feng W, He W, Yin F, Liu L (2018) Scene text detection with recurrent instance segmentation. 24th ICPR, 2227–2232

  • Grzegorzek M, Li C, Raskatow J, Paulus D, Vassilieva N (2013) Texture-Based text detection in digital images with wavelet features and Support Vector Machines. In: Proceedings of the 8th international conference on computer recognition systems CORES, pp 857–866

  • He K, Gkioxari G, Doll P, Girshick R (2017a) Mask R-CNN. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988

  • He W, Zhang X, Yin F, Liu C (2017b) Deep direct regression for multi-oriented scene text detection. In: Proceedings of the IEEE international conference on computer vision, pp 22–29

  • Hu H, Zhang C, Luo Y, Han J, Ding E (2017) WordSup: Exploiting word annotations for character based text detection. In: Proceedings of the IEEE international conference on computer vision, pp 4950–4959

  • Irvin RB, McKeown DM (1989) Methods for exploiting the relationship between buildings and their shadows in aerial imagery. IEEE Trans Syst Man Cybern 19(6):1564–1575

    Article  Google Scholar 

  • Jiang Y, Zhu X, Wang X, Yang S, Luo Z (2018) R2cnn: rotational region cnn for arbitrarily-oriented scene text detection. In: 2018 24th international conference on pattern recognition, pp 3610–3615

  • Katartzis A, Sahli H, Nyssen E, Cornelis J (2001) Detection of buildings from a single airborne image using a Markov random field model. In: IEEE 2001 international geoscience and remote sensing symposium 6: 2832–2834

  • Liao M, Zhu Z, Shi B, Xia G, BaiX (2018) Rotation-sensitive regression for oriented scene text detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5909–5918

  • Liow YT, Pavlidis T (1990) Use of shadows for extracting buildings in aerial images. Comput Vision, Graphics, Image Process 49(2):242–277

    Article  Google Scholar 

  • Liu X, Liang D, Yan S, Chen D, Yan J (2018) FOTS: Fast oriented text spotting with a unified network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5676–5685

  • Liu X, Zhou G, Zhang R, Wei X (2020) An accurate segmentation-based scene text detector with context attention and repulsive text border. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp 2344–2352

  • Long S, Ruan J, Zhang W, He X, Wu W, Yao C (2018) Textsnake: A flexible representation for detecting text of arbitrary shapes. In: Proceedings of the European conference on computer vision, pp 20–36

  • Lyu P, Yao C, Wu W, Yan S, Bai X (2018) Multi-oriented scene text detection via corner localization and region segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7553–7563

  • Nayef N, Yin F, Bizid I, Choi H, Feng Y, Karatzas D, Luo Z; Pal U, Rigaud C, Chazalon J, Khlif W, Luqman MM, Burie JC, Liu C, Ogier JM (2017) ICDAR2017 robust reading challenge on multi-lingual scene text detection and script identification-RRC-MLT. In: 2017 14th IAPR international conference on document analysis and recognition, pp 1454–1459

  • Perera C, Zaslavsky A, Christen P, Georgakopoulos D (2014) Sensing as a service model for smart cities supported by Internet of Things. T Emerg Telecommun T 25(1):81–93

    Google Scholar 

  • Qin S, Ren P, Kim S, Manduchi R (2018) Robust and accurate text stroke segmentation. In: 2018 IEEE winter conference on applications of computer vision, pp 242–250

  • Sanchez L, Munoz L, Galache JA, Sotres P, Santana JR, Gutierrez V, Ramdhan R, Gluhak A, KrcoS TE (2014) SmartSantander: IoT experimentation over a smart city testbed. Comput Netw 61:217–238

    Article  Google Scholar 

  • Shi B, Bai X, Belongie S (2017) Detecting oriented text in natural images by linking segments. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3482–3490

  • Stassopoulou A, Caelli T, Ramirez R (2000) Automatic extraction of building statistics from digital orthophotos. Int J Geogr Inf Sci 14(8):795–814

    Article  Google Scholar 

  • Van DN, Lu S, Bai X, Van, Ouarti, N, Mokhtari M (2017) Max-pooling based scene text proposal for scene text detection. In: 2017 14th IAPR international conference on document analysis and recognition, pp 1295–1300

  • Wang T, Wu DJ, Coates A, Ng AY (2012) End-to-end text recognition with convolutional neural networks. In: Proceedings of the 21st international conference on pattern recognition, pp 3304–3308

  • Wang W, Xie E, Li X, Hou, W, Lu, T, Yu, G (2019a) Shape robust text detection with progressive scale expansion network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 9328–9337

  • Wang X, Jiang Y, Luo Z, Liu CL, Choi H, Kim S (2019b) Arbitrary Shape Scene Text Detection with Adaptive Text Region Representation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6442–6451.

  • Zhang J, Zhang D, Bao M, Cheng J, Tang K (2016a) Traffic sign detection based on cascaded convolutional neural networks. In: 2016a 17th IEEE/ACIS international conference on software engineering, artificial intelligence, networking and parallel/distributed computing, pp 201–206

  • Zhang Z, Zhang C, Shen W, Yao C, Liu W, Bai X (2016b) Multi-oriented text detection with fully convolutional networks, In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4159–4167

  • Zhang C, Liang B, Huang Z, En M, Han J, Ding E, Ding X (2019) Look more than once: an accurate detector for text of arbitrary shapes. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 10544–10553

  • Zhong Z, Jin L, Huang S (2017a) DeepText: A new approach for text proposal generation and text detection in natural images. In: 2017a IEEE international conference on acoustics, speech and signal processing, pp 1208–1212

  • Zhong Z, Sun L, Huo Q (2017b) Improved localization accuracy by locnet for Faster R-CNN based text detection. In: 2017b 14th IAPR international conference on document analysis and recognition, pp 923–928

  • Zhou X, Yao C, Wen H, Wang Y, Zhou S, He W, Liang J (2017) East: an efficient and accurate scene text detector. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition pp 2642–2651

  • Zhu Y, Du J (2018) Sliding line point regression for shape robust scene text detection. In: 2018 24th international conference on pattern recognition, pp 3735–3740

Download references

Funding

This work was supported by the Chongqing Natural Science Foundation of China (Grant No. cstc2021jcyj-bsh0218), the Chongqing Science and Technology Bureau of China (Grant No. D63012021013), The National Natural Science Foundation of China (Grant No. U21A20447 and 61971079), The Basic Research and Frontier Exploration Project of Chongqing (Grant No. cstc2019jcyjmsxmX0666), Chongqing technological innovation and application development project (Grant No.cstc2021jscx-gksbx0051), The Innovative Group Project of the National Natural Science Foundation of Chongqing (Grant No. cstc2020jcyj-cxttX0002), and the Regional Creative Cooperation Program of Sichuan (Grant No.2020YFQ0025) and The Science and Technology Research Program of Chongqing Municipal Education Commission (Grant No.KJZD-k202000604).

Author information

Authors and Affiliations

  1. School of Economic Information Engineering, Southwestern University of Finance and Economics, Chengdu, 611130, China

    Yuanpeng Long

  2. Southwest Medical University, Luzhou, 646000, China

    Guo Zhang

  3. Chongqing University of Posts and Telecommunication, Chongqing, 400065, China

    Weiwei Sun, Yu Pang, Huiqian Wang & Guo Zhang

Authors
  1. Yuanpeng Long
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weiwei Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huiqian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guo Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Guo Zhang contributed to conceptualization; Guo Zhang and Yuanpeng Long contributed to methodology and guidance of the project; Yuanpeng Long and Guo Zhang contributed to validation, formal analysis and data analysis; Yuanpeng Long, Weiwei Sun, Yu Pang, Huiqian Wang and Guo Zhang contributed to writing.

Corresponding author

Correspondence to Guo Zhang.

Ethics declarations

Conflict of interest

The authors declare no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Long, Y., Sun, W., Pang, Y. et al. Research on text detection on building surfaces in smart cities based on deep learning. Soft Comput 26, 10103–10114 (2022). https://doi.org/10.1007/s00500-022-07391-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-022-07391-3

Keywords

Search

Navigation