Abstract
The recognition of hand-drawn chemical molecular formulas is crucial for applications such as electronic note-taking and automated grading. Despite the challenges posed by stylistic variations in hand-drawn chemical structure diagrams, we introduce a novel recognition algorithm for hand-drawn hydrocarbon molecular formulas using anchor-free object detection methods. First, we employ an anchor-free detector based on irregular quadrilaterals to identify all potential chemical bonds in input images. By analyzing the collision relationships between these bonds, we then reconstruct all unspecified carbon atoms and assemble them into an adjacency matrix. Finally, we use the RDKit to convert the adjacency matrix into a SMILES string. Notably, our method does not rely on the SMILES string used during training, thereby enabling it to recognize previously unseen hydrocarbons. To verify the effectiveness of the algorithm, we collected a dataset containing 4,217 hand-drawn hydrocarbon molecular structures. Using RepVGG-A0 at a \(512\,\times \,512\) resolution, our algorithm achieved a recognition accuracy of 85.86%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Clevert, D.A., Le, T., et al.: Img2mol - accurate smiles recognition from molecular graphical depictions. Chem. Sci. 12, 14174–14181 (2021)
Ding, X., Zhang, X., Ma, N., et al.: Repvgg: making VGG-style convnets great again. In: 2021 IEEE/CVF CVPR, pp. 13728–13737 (2021)
Duan, K., Xie, L., Qi, H., et al.: Location-sensitive visual recognition with cross-iou loss. arXiv preprint arXiv:2104.04899 (2021)
Fujiyoshi, A., Nakagawa, K., Suzuki, M.: Robust method of segmentation and recognition of chemical structure images in cheminfty. In: Pre-proceedings of the 9th IAPR International Workshop on Graphics Recognition (2011)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE/CVF CVPR, pp. 770–778 (2016)
Hu, J., Wu, H., Chen, M., et al.: Handwritten chemical structure image to structure-specific markup using random conditional guided decoder. In: Proceedings of the 31st ACM International Conference on Multimedia, 2023, Ottawa, pp. 8114–8124. ACM (2023)
Landrum, G.: Rdkit: open-source cheminformatics. Website (2010). https://www.rdkit.org
Lin, T.Y., Goyal, P., Girshick, R., et al.: Focal loss for dense object detection. In: 2017 IEEE ICCV, pp. 2999–3007 (2017)
Loshchilov, I., Hutter, F.: Sgdr: stochastic gradient descent with warm restarts. In: 5th ICLR, pp. 1–16 (2017)
Ma, N., Zhang, X., et al.: Shufflenet v2: practical guidelines for efficient CNN architecture design. arXiv preprint arXiv:1807.11164 pp. 1–19 (2018)
Tan, M., Pang, R., Le, Q.V.: Efficientdet: scalable and efficient object detection. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, 13–19 June 2020, pp. 10778–10787 (2020)
Weir, H., Thompson, K., Woodward, A., et al.: Chempix: automated recognition of hand-drawn hydrocarbon structures using deep learning. Chem. Sci. 12, 10622–10633 (2021)
Zhang, J., Tao, J.-J., Kuang, L.-D., Gui, Y.: Group residual dense block for key-point detector with one-level feature. In: Khanna, S., Cao, J., Bai, Q., Xu, G. (eds.) PRICAI 2022, Part II, pp. 525–539. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20865-2_39
Zhang, X., Yi, J., Yang, G., et al.: Abc-net: a divide-and-conquer based deep learning architecture for SMILES recognition from molecular images. Briefings Bioinform. 23(2) (2022)
Zhang, Y., Tian, Y., Kong, Y., et al.: Residual dense network for image super-resolution. In: 2018 IEEE/CVF CVPR, pp. 2472–2481 (2018)
Zheng, L., Zhang, T., Yu, X.: Recognition of handwritten chemical organic ring structure symbols using convolutional neural networks. In: 2019 ICDARW, pp. 165–168 (2019)
Zhou, X., Wang, D., Krähenbühl, P.: Centernet: objects as points. arXiv preprint arXiv:1904.07850 pp. 1–12 (2019)
Acknowledgments
This work was supported by the Scientific Research Fund of Hunan University of Chinese Medicine under Grant 2024XJZC006.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Tao, JJ., Liu, W., Peng, X., He, X., Luo, Y. (2025). Recognition of Hand-Drawn Hydrocarbon Structure Formulas Using Anchor-Free Detector. In: Hadfi, R., Anthony, P., Sharma, A., Ito, T., Bai, Q. (eds) PRICAI 2024: Trends in Artificial Intelligence. PRICAI 2024. Lecture Notes in Computer Science(), vol 15285. Springer, Singapore. https://doi.org/10.1007/978-981-96-0128-8_9
Download citation
DOI: https://doi.org/10.1007/978-981-96-0128-8_9
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-96-0127-1
Online ISBN: 978-981-96-0128-8
eBook Packages: Computer ScienceComputer Science (R0)