Skip to main content
SpringerLink
Log in

A coarse to fine framework for recognizing and locating multiple diatoms with highly complex backgrounds in forensic investigation

  • 1193: Intelligent Processing of Multimedia Signals
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In the forensic investigation, recognizing and locating the multiple diatom objects in an image is a challenging issue due to the interferences of the highly complex backgrounds. To address this issue, a coarse to fine diatom recognition and a localization framework based on the deep learning network is proposed in this paper. Firstly, the diatom images are obtained by performing the anatomic study on the cadavers. Next, a high definition electron microscope is scanned. Then, a coarse to fine deep learning framework is constructed to locate and recognize the diatom objects. Unlike the existing diatom classification and recognition methods, which used light microscopy with low resolution and completed under a simple backgrounds, our framework utilizes the high definition electron scanning microscopy with much higher resolution and suffers from the complex backgrounds interferences. To demonstrate the effectiveness of the proposed framework, 4 diatom image datasets with different background interference degrees are constructed. Also, 3 computer numerical simulation analysis are performed. They are (1) the limitations of the traditional methods in the diatom recognition, (2) the optimized composition of the training strategies and the network models, and (3) the performance of the proposed framework. The computer numerical simulation results show that the proposed framework achieves a recognition accuracy of 0.852. This is greater than 0.758 achieved by the AlexNet. Moreover, it can overcome the problem of the highly complex backgrounds interferences in the forensic investigation. Furthermore, it can locate and recognize the multiple objects in various diatom images simultaneously.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Chang L, Duarte MM, Sucar LE, Morales EF (2012) A Bayesian approach for object classification based on clusters of sift local features. Expert Syst Appl 39(2):1679–1686

    Article  Google Scholar 

  2. Chang CC, Lin CJ (2011) A Library for support vector machines. ACM Trans Intell Syst Technol 2(3):1–39

    Article  Google Scholar 

  3. Deng JH, Wang XY, Zhao J, Liu C, Kang XD, Gu GS (2019) Cyclotella recognition of high-resolution electron microscopy with complex backgrounds. Comput Eng Design 40:167–172

    Google Scholar 

  4. Fang WL, Ding LY, Zhong BT, Love PED, Luo HB (2018) Automated detection of workers and heavy equipment on construction sites: A convolutional neural network approach. Adv Eng Inform 37:139–149

    Article  Google Scholar 

  5. Fei-Fei L, Perona P (2005) A Bayesian hierarchical model for learning natural scene categories. 2005 IEEE Comput Soc Conf Comput Vision Patt Recognit (CVPR’05) 2:524–531

    Article  Google Scholar 

  6. Girshick R (2015) Fast r-cnn. In: Proceedings of the ieee international conference on computer vision, pp 1440–1448

  7. Girshick R, Donahue J, Darrell T, et al. (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587

  8. Krause LMK, Koc J, Rosenhahn B, et al. (2020) A fully convolutional neural network for detection and counting of diatoms on coatings after Short-Term field exposure. Environ Technol 54(16):10022–10030

    Article  Google Scholar 

  9. Krizhevsky A, Sutskever I, Hinton G (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inform Process Syst 25:1097–1105

    Google Scholar 

  10. Liu W, Anguelov D, Erhan D, et al. (2016) SSD: Single Shot MultiBox Detector. Computer Vision - ECCV 2016. ECCV 2016. Lecture Notes in Computer Science. Springer, Cham. https://doi.org/10.1007/978-3-319-46448-0_2

    Google Scholar 

  11. Liu WY, Wen YD, Yu ZD, Ment Y (2016) Large-margin softmax loss for convolutional neural networks. ICML 2(3):7

    Google Scholar 

  12. Ludes B, Coste M, North N, Doray S, Tracqui A, Kintz P (1999) Diatom analysis in victim’s tissues as an indicator of the site of drowning. Int J Legal Med 112(3):163–166

    Article  Google Scholar 

  13. Mikolajczyk K, Schmid C (2005) A performance evaluation of local descriptors. IEEE Trans Pattern Anal Mach Intell 27(10):1615–1630

    Article  Google Scholar 

  14. Pedraza A, Bueno G, Deniz O, Cristobal G, Blanco S, Borrego-Ramos M (2017) Automated diatom classification (part b): a deep learning approach. Appl Sci 7(5):460

    Article  Google Scholar 

  15. Pedraza A, Bueno G, Deniz O, Ruiz-Santaquiteria J, Sanchez C, Blanco S, Borrego-Ramos M, Olenici A, Cristobal G (2018) Lights and pitfalls of convolutional neural networks for diatom identification. Optics Photonics Digit Technol Imaging Appl 14

  16. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 779–788

  17. Redmon J, Farhadi A (2018) YOLOv3: An Incremental Improvement. arXiv e-prints

  18. Ren SQ, He KM, Girshick R, Sun J (2015) r-cnn: Faster Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  19. Simonyan K, Zisserman A (2014) Very Deep convolutional networks for large-scale image recognition. Comput Sci

  20. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: A simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958

    MathSciNet  MATH  Google Scholar 

  21. Uijlings J, Van D, Gevers T, Smeulsers A (2013) Selective search for object recognition. Int J Comput Vis 104(2):154–171

  22. Verikas A, Gelzinis A, Bacauskiene M, Olenina I, Olenin S, vaiciukynas E (2012) Phase congruency-based detection of circular objects applied to analysis of phytoplankton images. Patt Recognit 45(4):1659–167

    Article  Google Scholar 

  23. Xie JZ, Luo TW, Dai JW, wang D, Gao Y, Ran S (2013) Research on automatic analysis algorithm of streaming image of red tide algae. Comput Sci 40:293–296

    Google Scholar 

  24. Yu W, Xue Y, Knoops R, et al. (2020) Automated diatom searching in the digital scanning electron microscopy images of drowning cases using the deep neural networks. Int J Legal Med (1). https://doi.org/10.1007/s00414-020-02392-z

  25. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. European conference on computer vision 818–833

  26. Zhang YD, Wang SH, Sun P, Phillips P (2015) Pathological brain detection based on wavelet entropy and hu moment invariants. Bio-Med Mater Eng 26:S1283–S1290

    Article  Google Scholar 

  27. Zhang Q, Zhang M, Chen T, et al. (2018) Recent advances in convolutional neural network acceleration. Neurocomputing

  28. Zhang S, Zhou Y, Zhang QZ (2016) Application of the word package model in algae classification and recognition. J Beijing Inform Sci Technol Univ 31:28–32

    Google Scholar 

  29. Zhao J, Liu C, Hu S, et al. (2013) Microwave digestion-vacuum filtration-automated scanning electron microscopy as a sensitive method for forensic diatom test. Int J Legal Med 127(2):459–463

    Article  Google Scholar 

  30. Zhong Z, Sun L, Huo Q (2017) Improved localization accuracy by locnet for faster r-cnn based text detection. In: 2017 14th IAPR international conference on document analysis and recognition (ICDAR). IEEE Computer Society, pp 923–928

  31. Zhou Y, Zhang J, Huang J, et al. (2019) Digital whole-slide image analysis for automated diatom test in forensic cases of drowning using a convolutional neural network algorithm. Forens Sci Int 302:109922

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the Key Areas Research and Development Program of Guangdong Province under Grant 2019B010139002, the project of Guangzhou Science and Technology under Grant 201902020006, 201902020007, 201807010058.

Author information

Authors and Affiliations

  1. School of Computers, Guangdong University of Technology, Guangzhou, China

    Jiehang Deng, Haomin Wei, Dongdong He, Guosheng Gu, Hongjin Liang, Peijie Wu & Yuanli Zhong

  2. School of Computer Science and Software, Zhaoqing University, Zhaoqing, China

    Jiehang Deng

  3. Guangzhou Forensic Science Institute and Key Laboratory of Forensic Pathology, Ministry of Public Security, Guangzhou, China

    Xiaodong Kang, Chao Liu & Jian Zhao

  4. School of Mathematics and Statistics, Zhaoqing University, Zhaoqing, China

    Shihe Xu

  5. School of Information Engineering, Guangdong University of Technology, Guangzhou, China

    Wing-Kuen Ling

Authors
  1. Jiehang Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haomin Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dongdong He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guosheng Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaodong Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hongjin Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Chao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Peijie Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Yuanli Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Shihe Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Wing-Kuen Ling
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Jian Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guosheng Gu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Deng, J., Wei, H., He, D. et al. A coarse to fine framework for recognizing and locating multiple diatoms with highly complex backgrounds in forensic investigation. Multimed Tools Appl 81, 4839–4857 (2022). https://doi.org/10.1007/s11042-021-11169-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11169-4

Keywords

Search

Navigation