Abstract
The identification of graphic symbols and interconnections is a primary task in the digitization of symbolic engineering diagram images like circuit diagrams. Recent approaches propose the use of Convolutional Neural Networks to the identification of symbols in engineering diagrams. Although recall and precision from CNN based object recognition algorithms are high, false negatives result in some input symbols being missed or misclassified. The missed symbols induce errors in the circuit level features of the extracted circuit, which can be identified using graph level analysis. In this work, a custom annotated printed circuit image set, which is made publicly available in conjunction with the source code of the experiments of this paper, is used to fine-tune a Faster RCNN network to recognise component symbols and blob detection to identify inter-connections between symbols to generate a graph representation of the extracted circuit components. The graph structure is then analysed using graph convolutional neural networks and node degree comparison to identify graph anomalies potentially resulting from false negatives from the object recognition module. Anomaly predictions are then used to identify image regions with potential missed symbols, which are subject to image transforms and re-input to the Faster RCNN, which results in a significant improvement in component recall, which increases to 91% on the test set. The general tools used by the analysis pipeline can also be applied to other Engineering Diagrams with the availability of similar datasets.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Agarwal, S., Agrawal, M., Chaudhury, S.: Recognizing electronic circuits to enrich web documents for electronic simulation. In: Lamiroy, B., Dueire Lins, R. (eds.) GREC 2015. LNCS, vol. 9657, pp. 60–74. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-52159-6_5
Bailey, D., Norman, A., Moretti, G., North, P.: Electronic schematic recognition. Massey University, Wellington, New Zealand (1995)
Bayer, J., Sinha, A.: Graph-based manipulation rules for piping and instrumentation diagrams (2020)
De, P., Mandal, S., Bhowmick, P.: Recognition of electrical symbols in document images using morphology and geometric analysis. In: 2011 International Conference on Image Information Processing, pp. 1–6 (2011)
Drapeau, J., Géraud, T., Coustaty, M., Chazalon, J., Burie, J.-C., Eglin, V., Bres, S.: Extraction of ancient map contents using trees of connected components. In: Fornés, A., Lamiroy, B. (eds.) GREC 2017. LNCS, vol. 11009, pp. 115–130. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-02284-6_9
Fu, L., Kara, L.B.: From engineering diagrams to engineering models: visual recognition and applications. Comput. Aided Des. 43(3), 278–292 (2011)
Hammond, D.K., Vandergheynst, P., Gribonval, R.: Wavelets on graphs via spectral graph theory. Appl. Comput. Harmon. Anal. 30(2), 129–150 (2011)
Héroux, P., Le Bodic, P., Adam, S.: Datasets for the evaluation of substitution-tolerant subgraph isomorphism. In: Lamiroy, B., Ogier, J.-M. (eds.) GREC 2013. LNCS, vol. 8746, pp. 240–251. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44854-0_19
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Maier, A.: Graph deep learning — part 1. https://towardsdatascience.com/graph-deep-learning-part-1-e9652e5c4681
Messmer, B.T., Bunke, H.: Automatic learning and recognition of graphical symbols in engineering drawings. In: Kasturi, R., Tombre, K. (eds.) GREC 1995. LNCS, vol. 1072, pp. 123–134. Springer, Heidelberg (1996). https://doi.org/10.1007/3-540-61226-2_11
Moreno-García, C.F., Elyan, E., Jayne, C.: New trends on digitisation of complex engineering drawings. Neural Comput. Appl. 31(6), 1695–1712 (2018). https://doi.org/10.1007/s00521-018-3583-1
Rabbani, M., Khoshkangini, R., Nagendraswamy, H., Conti, M.: Hand drawn optical circuit recognition. Procedia Comput. Sci. 84, 41–48 (2016). Proceeding of the Seventh International Conference on Intelligent Human Computer Interaction (IHCI 2015). http://www.sciencedirect.com/science/article/pii/S1877050916300783
Rahul, R., Paliwal, S., Sharma, M., Vig, L.: Automatic information extraction from piping and instrumentation diagrams. arXiv preprint arXiv:1901.11383 (2019)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2016)
Renton, G., Héroux, P., Gaüzère, B., Adam, S.: Graph neural network for symbol detection on document images. In: 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW), vol. 1, pp. 62–67. IEEE (2019)
Surikov, I.Y., Nakhatovich, M.A., Belyaev, S.Y., Savchuk, D.A.: Floor plan recognition and vectorization using combination UNet, faster-RCNN, statistical component analysis and ramer-douglas-peucker. In: Chaubey, N., Parikh, S., Amin, K. (eds.) COMS2 2020. CCIS, vol. 1235, pp. 16–28. Springer, Singapore (2020). https://doi.org/10.1007/978-981-15-6648-6_2
Valveny, E., Delalandre, M., Raveaux, R., Lamiroy, B.: Report on the symbol recognition and spotting contest. In: Kwon, Y.-B., Ogier, J.-M. (eds.) GREC 2011. LNCS, vol. 7423, pp. 198–207. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36824-0_19
Xiong, X., Choi, B.J.: Comparative analysis of detection algorithms for corner and blob features in image processing. Int. J. Fuzzy Logic Intell. Syst. 13(4), 284–290 (2013)
Yu, B.: Automatic understanding of symbol-connected diagrams. In: Proceedings of 3rd International Conference on Document Analysis and Recognition, vol. 2, pp. 803–806. IEEE (1995)
Yu, E.S., Cha, J.M., Lee, T., Kim, J., Mun, D.: Features recognition from piping and instrumentation diagrams in image format using a deep learning network. Energies 12(23), 4425 (2019)
Yun, D.Y., Seo, S.K., Zahid, U., Lee, C.J.: Deep neural network for automatic image recognition of engineering diagrams. Appl. Sci. 10(11), 4005 (2020)
Zhou, X., et al.: East: an efficient and accurate scene text detector. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5551–5560 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Mizanur Rahman, S., Bayer, J., Dengel, A. (2021). Graph-Based Object Detection Enhancement for Symbolic Engineering Drawings. In: Barney Smith, E.H., Pal, U. (eds) Document Analysis and Recognition – ICDAR 2021 Workshops. ICDAR 2021. Lecture Notes in Computer Science(), vol 12916. Springer, Cham. https://doi.org/10.1007/978-3-030-86198-8_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-86198-8_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86197-1
Online ISBN: 978-3-030-86198-8
eBook Packages: Computer ScienceComputer Science (R0)