Skip to main content
SpringerLink
Log in

A benchmark dataset in chemical apparatus: recognition and detection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Robots that perform chemical experiments autonomously have been implemented, using the same chemical apparatus as human chemists and capable of performing complex chemical experiments unmanaged. However, most robots in chemistry are still programmed and cannot adapt to diverse environments or to changes in displacement and angle of the object. To resolve this issue, we have conceived a computer vision method for identifying and detecting chemical apparatus automatically. Identifying and localizing such apparatus accurately from chemistry lab images is the most important task. We acquired 2246 images from real chemistry laboratories, with a total of 33,108 apparatus instances containing 21 classes. We demonstrate a Chemical Apparatus Benchmark Dataset (CABD) containing a chemical apparatus image recognition dataset and a chemical apparatus object detection dataset. We evaluated five excellent image recognition models: AlexNet, VGG16, GoogLeNet, ResNet50, MobileNetV2 and four state-of-the-art object detection methods: Faster R-CNN (3 backbones), Single Shot MultiBox Detector (SSD), YOLOv3-SPP and YOLOv5, respectively, on the CABD dataset. The results can serve as a baseline for future research. Experiments show that ResNet50 has the highest accuracy (99.9%) in the chemical apparatus image recognition dataset; Faster R-CNN (ResNet50-fpn) and YOLOv5 performed the best in terms of mAP (99.0%) and AR (94.5%) in the chemical apparatus object detection dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

The data that support the findings of this study are openly available in Zenodo,reference number [49].

Notes

  1. An earlier version of this paper was presented at the International conference on Artificial Intelligence and Big Data in Digital Era.

References

  1. Ding Z, Ran S, Wu Z et al  (2022) A new benchmark data set for Chemical laboratory apparatus detection[C]. Artificial Intelligence in Data and Big Data Processing: Proceedings of ICABDE, pp 201–210

  2. Kelley EW (2021) Sample plan for easy, inexpensive, safe, and relevant hands-on, aat-home wet organic Chemistry laboratory activities[J]. J Chem Educ 98(5):1622–1635

    Article  ADS  CAS  Google Scholar 

  3. Willey RJ, Carter T, Price J et al (2020) Instruction of hazard analysis of methods for chemical process safety at the university level[J]. J Loss Prev Process Ind 63:103961

    Article  CAS  Google Scholar 

  4. Christensen H, Amato N, Yanco H et al (2021) A roadmap for us robotics–from internet to robotics 2020 edition[J]. Foundations and Trends®. Robotics 8(4):307–424

    Google Scholar 

  5. Althoff M, Giusti A, Liu SB et al (2019) Effortless creation of safe robots from modules through self-programming and self-verification[J]. Science. Robotics 4(31):eaaw1924

    Article  Google Scholar 

  6. Zoph B, Vasudevan V, Shlens J, et al. (2018) Learning transferable architectures for scalable image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8697–8710

  7. Liu L, Ouyang W, Wang X (2020) Deep learning for generic object detection: a survey)[J]. Int J Comput Vis 128(2):261–318

    Article  Google Scholar 

  8. Kumar A, Zhou Y, Gandhi CP et al (2020) Bearing defect size assessment using wavelet transform based Deep Convolutional Neural Network (DCNN)[J]. Alexandria Eng J 59(2):999–1012

    Article  Google Scholar 

  9. Frizzi S, Kaabi R, Bouchouicha M et al (2016) Convolutional neural network for video fire and smoke detection[C]//IECON 2016-42nd Annual Conference of the IEEE Industrial Electronics Society. IEEE:877–882

  10. Knickmeyer D (2020) Social factors influencing household waste separation: A literature review on good practices to improve the recycling performance of urban areas[J]. J Clean Prod 245:118605

    Article  Google Scholar 

  11. Cao W, Liu Q, He Z (2020) Review of pavement defect detection methods[J]. IEEE Access 8:14531–14544

    Article  Google Scholar 

  12. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks[J]. Adv Neural Inf Proces Syst 25

  13. Simonyan K, Zisserman A. (2014) Very deep convolutional networks for large-scale image recognition[J]. ArXiv Preprint ArXiv:1409.1556

  14. Khan RU, Zhang X, Kumar R (2019) Analysis of ResNet and GoogleNet models for malware detection[J]. J Comput Virolog Hacking Techniq 15(1):29–37

    Article  Google Scholar 

  15. Mukti IZ, Biswas D (2019) Transfer learning based plant diseases detection using ResNet50[C]//2019 4th International Conference on Electrical Information and Communication Technology (EICT). IEEE:1–6

  16. Sandler M, Howard A, Zhu M et al (2018) Mobilenetv2: Inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4510–4520

  17. Ren S, He K, Girshick R (2015) Faster R-cnn: towards real-time object detection with region proposal networks[J]. Adv Neural Inf Proces Syst 28:91–99

    Google Scholar 

  18. Huang Z, Wang J, Fu X (2020) DC-SPP-YOLO: Dense connection and spatial pyramid pooling based YOLO for object detection[J]. Inf Sci 522:241–258

    Article  MathSciNet  Google Scholar 

  19. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg A C (2016) SSD: Single shot multibox detector[C]. In: European Conference on Computer Vision. Springer, Cham, pp 21–37

  20. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation[C]. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 580–587

  21. Wang H, Yuan H, Hong SS et al (2015) Physical and chemical tuning of two-dimensional transition metal dichalcogenides[J]. Chem Soc Rev 44(9):2664–2680

    Article  CAS  PubMed  Google Scholar 

  22. Zanchettin AM, Ceriani NM, Rocco P et al (2015) Safety in human-robot collaborative manufacturing environments: Metrics and control[J]. IEEE Trans Autom Sci Eng 13(2):882–893

    Article  Google Scholar 

  23. Henson AB, Gromski PS, Cronin L (2018) Designing algorithms to aid discovery by chemical robots[J]. ACS Centr Sci 4(7):793–804

    Article  CAS  Google Scholar 

  24. Zhavoronkov A (2018) Artificial intelligence for drug discovery, biomarker development, and generation of novel chemistry[J]. Mol Pharm 15(10):4311–4313

    Article  CAS  PubMed  Google Scholar 

  25. Burger B, Maffettone PM, Gusev VV et al (2020) A mobile robotic chemist[J]. Nature 583(7815):237–241

    Article  ADS  CAS  PubMed  Google Scholar 

  26. Patrício DI, Rieder R (2018) Computer vision and artificial intelligence in precision agriculture for grain crops: A systematic review[J]. Comput Electron Agric 153:69–81

    Article  Google Scholar 

  27. Chai J, Li A (2019) Deep learning in natural language processing: A state-of-the-art survey[C]//2019 International Conference on Machine Learning and Cybernetics (ICMLC). IEEE:1–6

  28. Vrancken C, Longhurst P, Wagland S (2019) Deep learning in material recovery: Development of method to create training database[J]. Expert Syst Appl 125:268–280

    Article  Google Scholar 

  29. Jadon A, Omama M, Varshney A, et al. (2019) FireNet: a specialized lightweight fire & smoke detection model for real-time IoT applications[J]. arXiv preprint arXiv:1905.11922

  30. Nguyen A, Yosinski J, Clune J (2015) Deep neural networks are easily fooled: High confidence predictions for unrecognizable images[C]. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 427–436

  31. Savadjiev P, Chong J, Dohan A et al (2019) Demystification of AI-driven medical image interpretation: past, present and future[J]. Eur Radiol 29(3):1616–1624

    Article  PubMed  Google Scholar 

  32. Fujiyoshi H, Hirakawa T, Yamashita T (2019) Deep learning-based image recognition for autonomous driving[J]. IATSS Res 43(4):244–242

    Article  Google Scholar 

  33. Parmar DN, Mehta BB (2014) Face recognition methods & applications[J]. arXiv preprint arXiv:1403.0485

  34. Pan X, Shi J, Luo P et al (2018) Spatial as deep: Spatial CNN for traffic scene understanding[C]. Proceedings of the AAAI Conference on Artificial Intelligence 32(1):7276–7283

  35. Lu ZM, Li SZ, Burkhardt H (2006) A content-based image retrieval scheme in JPEG compressed domain[J]. Int J Innov Comput, Inform Contr 2(4):831–839

    Google Scholar 

  36. Li LF, Wang X, Hu WJ et al (2020) Deep learning in skin disease image recognition: A review[J]. IEEE Access 8:208264–208280

    Article  Google Scholar 

  37. Ma N, Zhang X, Zheng HT et al (2018) Shufflenet v2: Practical guidelines for efficient CNN architecture design[C]. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 116–131

  38. Tan M, Le Q (2021) Efficientnetv2: Smaller models and faster training[C]//International Conference on Machine Learning. PMLR:10096–10106

  39. Hossain S, Lee D (2019) Deep learning-based real-time multiple-object detection and tracking from aerial imagery via a flying robot with GPU-based embedded devices[J]. Sensors 19(15):3371

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  40. Du L, Zhang R, Wang X (2020) Overview of two-stage object detection algorithms[C]//Journal of Physics: Conference Series. IOP Publishing 1544(1):012033

    Google Scholar 

  41. Girshick R, Donahue J, Darrell T, et al. (2014) Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580–587

  42. Girshick R (2015) Fast R-cnn[C]. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1440–1448

  43. He K, Gkioxari G, Dollár P, Girshick R (2017) Mask R-cnn[C]. In: Proceedings of the IEEE International Conference on Computer Vision, pp 2961–2969

  44. Lin TY, Goyal P, Girshick R (2017) Focal loss for dense object detection[C]. In: Proceedings of the IEEE International Conference on Computer Vision, pp 2980–2988

  45. Batth RS, Nayyar A, Nagpal A (2018) Internet of robotic things: driving intelligent robotics of future-concept, architecture, applications and technologies[C]//2018 4th International Conference on Computing Sciences (ICCS). IEEE:151–160

  46. Wan S, Gu Z, Ni Q (2020) Cognitive computing and wireless communications on the edge for healthcare service robots[J]. Comput Commun 149:99–106

    Article  Google Scholar 

  47. Rostianingsih S, Setiawan A, Halim CI (2020) COCO (creating common object in context) dataset for chemistry apparatus[J]. Procedia Comput Sci 171:2445–2452

    Article  Google Scholar 

  48. Deng J, Dong W, Socher R et al (2009) Imagenet: A large-scale hierarchical image database[C]. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp 248–255

  49. Ding Z (2023) Chemical Apparatus Benchmark Dataset: CABD, Zenodo.org, https://doi.org/10.5281/zenodo.7919476.

  50. Shetty S (2016) Application of convolutional neural network for image classification on Pascal VOC challenge 2012 dataset[J]. arXiv preprint arXiv:1607.03785

  51. Zhang Q, Zhang X, Mu X et al (2021) Recyclable waste image recognition based on deep learning[J]. Resour Conserv Recycl 171:105636

    Article  Google Scholar 

  52. Varga D (2020) A combined full-reference image quality assessment method based on convolutional activation maps[J]. Algorithms 13(12):313

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was supported by the grant of Anhui Provincial Natural Science Foundation, Nos. 1908085MF184, 1908085QF285, the grant of Scientific Research and Talent Development Foundation of the Hefei University, No.21-22RC15, the Key Research Plan of Anhui Province, Nos. 202104d07020006, 2022 k07020011, the grant of the Hefei University Postgraduate Innovation and Entrepreneurship Project, Nos. 21YCXL16,21YCXL14, in part by the grant of Key Generic Technology Research and Development Project of Hefei, No. 2021GJ030, the grant of Program for Scientific Research Innovation Team in Colleges and Universities of Anhui Province 2022AH010095, as well as the AI General Computing Platform of Hefei University.

Author information

Authors and Affiliations

  1. School of Artificial Intelligence and Big Data, Hefei University, Hefei, 230601, China

    Le Zou, Ze-Sheng Ding, Zhi-Huang He & Xiao-Feng Wang

  2. Institute of Applied Optimization, School of Artificial Intelligence and Big Data, Hefei University, Hefei, 230601, Anhui, China

    Shuo-Yi Ran & Zhi-Ze Wu

  3. School of Energy Materials and Chemical Engineering, Hefei University, Hefei, 230601, Anhui, China

    Yun-Sheng Wei

Authors
  1. Le Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ze-Sheng Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuo-Yi Ran
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhi-Ze Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yun-Sheng Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhi-Huang He
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xiao-Feng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiao-Feng Wang.

Ethics declarations

Conflict of interest

All authors declare that there are no conflicts of interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zou, L., Ding, ZS., Ran, SY. et al. A benchmark dataset in chemical apparatus: recognition and detection. Multimed Tools Appl 83, 26419–26437 (2024). https://doi.org/10.1007/s11042-023-16563-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16563-8

Keywords

Search

Navigation