Skip to main content

Graph-Based Object Detection Enhancement for Symbolic Engineering Drawings

  • Conference paper
  • First Online:
Document Analysis and Recognition – ICDAR 2021 Workshops (ICDAR 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12916))

Included in the following conference series:

Abstract

The identification of graphic symbols and interconnections is a primary task in the digitization of symbolic engineering diagram images like circuit diagrams. Recent approaches propose the use of Convolutional Neural Networks to the identification of symbols in engineering diagrams. Although recall and precision from CNN based object recognition algorithms are high, false negatives result in some input symbols being missed or misclassified. The missed symbols induce errors in the circuit level features of the extracted circuit, which can be identified using graph level analysis. In this work, a custom annotated printed circuit image set, which is made publicly available in conjunction with the source code of the experiments of this paper, is used to fine-tune a Faster RCNN network to recognise component symbols and blob detection to identify inter-connections between symbols to generate a graph representation of the extracted circuit components. The graph structure is then analysed using graph convolutional neural networks and node degree comparison to identify graph anomalies potentially resulting from false negatives from the object recognition module. Anomaly predictions are then used to identify image regions with potential missed symbols, which are subject to image transforms and re-input to the Faster RCNN, which results in a significant improvement in component recall, which increases to 91% on the test set. The general tools used by the analysis pipeline can also be applied to other Engineering Diagrams with the availability of similar datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/msyed-unikl/GraphFix.

  2. 2.

    https://github.com/msyed-unikl/printerd-circuit-ED-dataset.

  3. 3.

    discovercircuits.com.

  4. 4.

    LabelImg - https://github.com/tzutalin/labelImg.

  5. 5.

    https://pytorch.org/vision/stable/models.html#object-detection-instance-segmentation-and-person-keypoint-detection.

References

  1. Agarwal, S., Agrawal, M., Chaudhury, S.: Recognizing electronic circuits to enrich web documents for electronic simulation. In: Lamiroy, B., Dueire Lins, R. (eds.) GREC 2015. LNCS, vol. 9657, pp. 60–74. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-52159-6_5

    Chapter  Google Scholar 

  2. Bailey, D., Norman, A., Moretti, G., North, P.: Electronic schematic recognition. Massey University, Wellington, New Zealand (1995)

    Google Scholar 

  3. Bayer, J., Sinha, A.: Graph-based manipulation rules for piping and instrumentation diagrams (2020)

    Google Scholar 

  4. De, P., Mandal, S., Bhowmick, P.: Recognition of electrical symbols in document images using morphology and geometric analysis. In: 2011 International Conference on Image Information Processing, pp. 1–6 (2011)

    Google Scholar 

  5. Drapeau, J., Géraud, T., Coustaty, M., Chazalon, J., Burie, J.-C., Eglin, V., Bres, S.: Extraction of ancient map contents using trees of connected components. In: Fornés, A., Lamiroy, B. (eds.) GREC 2017. LNCS, vol. 11009, pp. 115–130. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-02284-6_9

    Chapter  Google Scholar 

  6. Fu, L., Kara, L.B.: From engineering diagrams to engineering models: visual recognition and applications. Comput. Aided Des. 43(3), 278–292 (2011)

    Article  Google Scholar 

  7. Hammond, D.K., Vandergheynst, P., Gribonval, R.: Wavelets on graphs via spectral graph theory. Appl. Comput. Harmon. Anal. 30(2), 129–150 (2011)

    Article  MathSciNet  Google Scholar 

  8. Héroux, P., Le Bodic, P., Adam, S.: Datasets for the evaluation of substitution-tolerant subgraph isomorphism. In: Lamiroy, B., Ogier, J.-M. (eds.) GREC 2013. LNCS, vol. 8746, pp. 240–251. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44854-0_19

    Chapter  Google Scholar 

  9. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)

  10. Maier, A.: Graph deep learning — part 1. https://towardsdatascience.com/graph-deep-learning-part-1-e9652e5c4681

  11. Messmer, B.T., Bunke, H.: Automatic learning and recognition of graphical symbols in engineering drawings. In: Kasturi, R., Tombre, K. (eds.) GREC 1995. LNCS, vol. 1072, pp. 123–134. Springer, Heidelberg (1996). https://doi.org/10.1007/3-540-61226-2_11

    Chapter  Google Scholar 

  12. Moreno-García, C.F., Elyan, E., Jayne, C.: New trends on digitisation of complex engineering drawings. Neural Comput. Appl. 31(6), 1695–1712 (2018). https://doi.org/10.1007/s00521-018-3583-1

    Article  Google Scholar 

  13. Rabbani, M., Khoshkangini, R., Nagendraswamy, H., Conti, M.: Hand drawn optical circuit recognition. Procedia Comput. Sci. 84, 41–48 (2016). Proceeding of the Seventh International Conference on Intelligent Human Computer Interaction (IHCI 2015). http://www.sciencedirect.com/science/article/pii/S1877050916300783

    Article  Google Scholar 

  14. Rahul, R., Paliwal, S., Sharma, M., Vig, L.: Automatic information extraction from piping and instrumentation diagrams. arXiv preprint arXiv:1901.11383 (2019)

  15. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2016)

    Article  Google Scholar 

  16. Renton, G., Héroux, P., Gaüzère, B., Adam, S.: Graph neural network for symbol detection on document images. In: 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW), vol. 1, pp. 62–67. IEEE (2019)

    Google Scholar 

  17. Surikov, I.Y., Nakhatovich, M.A., Belyaev, S.Y., Savchuk, D.A.: Floor plan recognition and vectorization using combination UNet, faster-RCNN, statistical component analysis and ramer-douglas-peucker. In: Chaubey, N., Parikh, S., Amin, K. (eds.) COMS2 2020. CCIS, vol. 1235, pp. 16–28. Springer, Singapore (2020). https://doi.org/10.1007/978-981-15-6648-6_2

    Chapter  Google Scholar 

  18. Valveny, E., Delalandre, M., Raveaux, R., Lamiroy, B.: Report on the symbol recognition and spotting contest. In: Kwon, Y.-B., Ogier, J.-M. (eds.) GREC 2011. LNCS, vol. 7423, pp. 198–207. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36824-0_19

    Chapter  Google Scholar 

  19. Xiong, X., Choi, B.J.: Comparative analysis of detection algorithms for corner and blob features in image processing. Int. J. Fuzzy Logic Intell. Syst. 13(4), 284–290 (2013)

    Article  Google Scholar 

  20. Yu, B.: Automatic understanding of symbol-connected diagrams. In: Proceedings of 3rd International Conference on Document Analysis and Recognition, vol. 2, pp. 803–806. IEEE (1995)

    Google Scholar 

  21. Yu, E.S., Cha, J.M., Lee, T., Kim, J., Mun, D.: Features recognition from piping and instrumentation diagrams in image format using a deep learning network. Energies 12(23), 4425 (2019)

    Article  Google Scholar 

  22. Yun, D.Y., Seo, S.K., Zahid, U., Lee, C.J.: Deep neural network for automatic image recognition of engineering diagrams. Appl. Sci. 10(11), 4005 (2020)

    Article  Google Scholar 

  23. Zhou, X., et al.: East: an efficient and accurate scene text detector. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5551–5560 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Johannes Bayer .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mizanur Rahman, S., Bayer, J., Dengel, A. (2021). Graph-Based Object Detection Enhancement for Symbolic Engineering Drawings. In: Barney Smith, E.H., Pal, U. (eds) Document Analysis and Recognition – ICDAR 2021 Workshops. ICDAR 2021. Lecture Notes in Computer Science(), vol 12916. Springer, Cham. https://doi.org/10.1007/978-3-030-86198-8_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86198-8_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86197-1

  • Online ISBN: 978-3-030-86198-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics