Skip to main content
SpringerLink
Log in

Deep learning based approach for digitized herbarium specimen segmentation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

As herbarium specimens are largely digitized and freely available in online portals, botanists aim to examine their taxonomic aspects to identify the plant specimen regions and generate their morphological data. Nevertheless, different uninformative visual information within the digitized herbarium specimen, such as scale-bar, color pallet, specimen label, envelopes, bar-code, and stamp, represent a source of visual noise. Thus, their identification requires unique detection methods as they are mostly placed at different locations and orientations within the herbarium sheet. Given a collection of digitized herbarium specimen images gathered from the Herbarium Haussknecht of Jena, Germany, we present in this paper a deep learning-based approach for specimen image semantic segmentation. Two different pipelines were involved in this work: (i) coarse segmentation and (ii) fine segmentation. Throughout the whole process, we describe the ground truth annotation used for training our deep learning architecture. The experimental results demonstrate that our proposed model outperforms the other architectures such as SegNet, Squeeze-SegNet, U-Net, and DeepLabv3. Its accuracy achieves 91% compared to 82%, 80%, 86%, and 90% obtained by SegNet, Squeeze-SegNet, U-Net, and DeepLabv3, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. https://spnhc.biowikifarm.net/wiki/Digitization

  2. https://fusion.cs.uni-jena.de/fusion/projects/managing-multimedia-data-for-science-mamuds/

  3. https://www.herbarium.uni-jena.de/

References

  1. Badrinarayanan V, Kendall A, Cipolla R (1 Dec. 2017) SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615

    Article  Google Scholar 

  2. Beghin T, Cope JS, Remagnino P, Barman S (2010) Shape and Texture Based Plant Leaf Classification. In: Blanc-Talon J, Bone D, Philips W, Popescu D, Scheunders P (eds) Advanced Concepts for Intelligent Vision Systems. ACIVS 2010. Lecture notes in computer science, vol 6475. Springer, Berlin. https://doi.org/10.1007/9783-642-17691-3-32

    Chapter  Google Scholar 

  3. Borges LM, Reis VC, Izbicki R (2020) Schr¨odinger’s phenotypes: herbarium specimens show two-dimensional images are both good and (not so) bad sources of morphological data. Methods Ecol Evol 11:1296–1308. https://doi.org/10.1111/2041-210X.13450

    Article  Google Scholar 

  4. Borsch T, Stevens A-D, Häffner E, Güntsch A, Berendsohn WG, Appelhans MS, Barilaro C, Beszteri B, Blattner FR, Bossdorf O, Dalitz H, Dressler S, Duque-Thüs R, Esser H-J, Franzke A, Goetze D, Grein M, Grünert U, Hellwig F, … Zizka G (2020) A complete digitization of German herbaria is possible, sensible and should be started now. Res Ideas Outcomes 6:e50675. https://doi.org/10.3897/rio.6.e50675

    Article  Google Scholar 

  5. Canny J, (1986) “A Computational Approach to Edge Detection,” in IEEE Trans Pattern Anal Mach Intell, vol. PAMI-8, no. 6, pp. 679–698, https://doi.org/10.1109/TPAMI.1986.4767851.

  6. Caselles V, Kimmel R, Guillermo Sapiro G (1997) Geodesic Active Contours. Int J Comput Vis 22(61–79):61–79

    Article  Google Scholar 

  7. Cerutti G, Tougne L, Mille J, Vacavant A, Coquin D (2013) Understanding leaves in natural images – a model-based approach for tree species identification. Comput Vis Image Underst 117(10):1482–1501. https://doi.org/10.1016/j.cviu.2013.07.003 ISSN 1077-3142, http://www.sciencedirect.com/science/article/pii/S1077314213001331

    Article  Google Scholar 

  8. Chen L, Papandreou G, Schroff F, Adam H (2017) Rethinking atrous convolution for semantic image segmentation. ArXiv, abs/1706.05587

  9. Figueira R, Lages F (2019) Museum and Herbarium Collections for Biodiversity Research in Angola. Springer International Publishing. https://doi.org/10.1007/978-3-030-03083-4-19

  10. Kommineni VK, Kattge J, Gaikwad J, Baddam P, Tautenhahn S (2020) Understanding intraspecific trait variability using digital herbarium specimen images. Biodivers Inf Sci Stand 4:e59061. https://doi.org/10.3897/biss.4.59061

    Article  Google Scholar 

  11. Kumar N, Belhumeur PN, Biswas A, Jacobs DW, Kress WJ, Lopez IC, Soares JVB (2012) editor=“Fitzgibbon, Andrew and Lazebnik, Svetlana and Perona, Pietro and Sato, Yoichi and Schmid, Cordelia. Leafsnap: A Computer Vision System for Automatic Plant Species Identification. Springer Berlin Heidelberg.

  12. Lee JW, Chan Yoon Y (2019) Fine-Grained Plant Identification using wide and deep learning model 1. 2019 international conference on platform technology and service (PlatCon), Jeju, pp 1–5. https://doi.org/10.1109/PlatCon.2019.8669407

    Book  Google Scholar 

  13. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. 2015 IEEE conference on computer vision and pattern recognition (CVPR), Boston, pp 3431–3440. https://doi.org/10.1109/CVPR.2015.7298965

    Book  Google Scholar 

  14. Jiang X, Lou S, Scott PJ (2012) Morphological method for surface metrology and dimensional metrology based on the alpha shape. Meas Sci Technol 23:015003

  15. McAllister CA, McKain MR, Li M, Bookout B, Kellogg EA (2018) Specimen based analysis of morphology and the environment in ecologically dominant grasses: the power of the herbarium. Philos Trans R Soc B 374:20170403. https://doi.org/10.1098/rstb.2017.0403

    Article  Google Scholar 

  16. Nanfack G, Elhassouny A, Thami RO (2018) Squeeze-SegNet: a new fast deep convolutional neural network for semantic segmentation. International Conference on Machine Vision

  17. Noh H, Hong S, Han B (2015) Learning Deconvolution Network for Semantic Segmentation. 2015 IEEE international conference on computer vision (ICCV), Santiago, pp 1520–1528. https://doi.org/10.1109/ICCV.2015.178

    Book  Google Scholar 

  18. Olsen A, Konovalov DA, Philippa B, Ridd P, Wood JC, Johns J, Banks W, Girgenti B, Kenny O, Whinney J, Calvert B, Azghadi MR, White RD (2019) DeepWeeds: a multiclass weed species image dataset for deep learning. Sci Rep 9:2058. https://doi.org/10.1038/s41598-018-38343-3

    Article  Google Scholar 

  19. Ronneberger O., Fischer P., Brox T. (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab N., Hornegger J., Wells W., Frangi A. (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. MICCAI 2015. Lecture notes in computer science. vol 9351, Springer, Cham https://doi.org/10.1007/978-3-319-24574-4-28

  20. Simonyan K, Zisserman A (2015) Very Deep Convolutional Networks for LargeScale Image Recognition. CoRR abs/1409.1556: n. pag

  21. Singh V, Varsha, Misra AK (2015) Detection of unhealthy region of plant leaves using image processing and genetic algorithm. 2015 International Conference on Advances in Computer Engineering and Applications, Ghaziabad, pp 1028–1032. https://doi.org/10.1109/ICACEA.2015.7164858

    Book  Google Scholar 

  22. Suzuki S, Abe K (1985) Topological structural analysis of digitized binary images by border following. Comput Vision Graph Image Process 30(1):32–46. https://doi.org/10.1016/0734-189X(85)90016-7 ISSN 0734-189X, http://www.sciencedirect.com/science/article/pii/0734189X85900167

    Article  MATH  Google Scholar 

  23. Valliammal N, Geethalakshmi S (2012) Plant leaf segmentation using non linear K means clustering

  24. Vukadinovic D, Polder G (2015) Watershed and supervised classification based fully automated method for separate leaf segmentation

  25. Wäldchen J, Rzanny M, Seeland M, Mäder P (2018) Automated plant species identification—trends and future directions. PLoS Comput Biol 14(4):1–19

    Article  Google Scholar 

  26. Wang Q, Gao J, Yuan Y (2018) A Joint Convolutional Neural Networks and Context Transfer for Street Scenes Labeling. IEEE Trans Intell Transp Syst 19(5):1457–1470. https://doi.org/10.1109/TITS.2017.2726546

    Article  Google Scholar 

  27. Wang Q, Gao J, Li X (2019) Weakly Supervised Adversarial Domain Adaptation for Semantic Segmentation in Urban Scenes. IEEE Trans Image Process 28(9):4376–4386. https://doi.org/10.1109/TIP.2019.2910667

    Article  MathSciNet  MATH  Google Scholar 

  28. Weaver WN, Ng J, Laport RG (2020) LeafMachine: Using machine learning to automate leaf trait extraction from digitized herbarium specimens. Appl Plant Sci 8(6):e11367. https://doi.org/10.1002/aps3.11367

    Article  Google Scholar 

  29. Weaver WN, Ng J, Laport RG (2020) LeafMachine: Using machine learning to automate leaf trait extraction from digitized herbarium specimens. Appl Plant Sci 8(6):e11367. Published 2020 Jul 1. https://doi.org/10.1002/aps3.11367

    Article  Google Scholar 

  30. Wilf P, Zhang S, Chikkerur S, Little SA, Wing SL, Serre T (Mar 2016) Computer vision cracks the leaf code. Proc Natl Acad Sci 113(12):3305–3310. https://doi.org/10.1073/pnas.1524473113

    Article  Google Scholar 

  31. Yu F, Koltun V (2016) Multi-Scale Context Aggregation by Dilated Convolutions. CoRR abs/1511.07122: n. pags

  32. Zhao C, Chan SSF, Cham W-K, Chu LM (2015) Plant identification using leaf shapes—A pattern counting approach. Pattern Recogn 48(10):3203–3215. https://doi.org/10.1016/j.patcog.2015.04.004 ISSN 0031-3203, http://www.sciencedirect.com/science/article/pii/S0031320315001284

    Article  Google Scholar 

Download references

Acknowledgements

This work is part of the MAMUDS project (Management Multimedia Data for Science) funded by the German ministry of education and research (BMBF Project No. 01D16009) and the Tunisian ministry of higher education and research. We thank the herbarium Haussknecht of Jena for cooperating with them and providing the dataset to train our model. We also thank FSU Jena and Sfax University. The authors would like to thank Prof. Dr. Birgitta Konig-Ries, Prof.Dr.Frank Hellwig and Dr. Jorn Hentschel for supporting this project.

Funding

Supported by the German ministry of education and research (BMBF) and Tunisian ministry of higher education and research (MESR).

Author information

Authors and Affiliations

  1. MIRACL Laboratory Sfax, University of Sfax, Sfax, Tunisia

    Abdelaziz Triki, Bassem Bouaziz & Walid Mahdi

  2. Friedrich Schiller University, Jena, Germany

    Hamdi Hamed & Jitendra Gaikwad

Authors
  1. Abdelaziz Triki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bassem Bouaziz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Walid Mahdi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hamdi Hamed
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jitendra Gaikwad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abdelaziz Triki.

Ethics declarations

Conflict of interests

The authors declare that there is no conflict of interests regarding the publication of this article.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Triki, A., Bouaziz, B., Mahdi, W. et al. Deep learning based approach for digitized herbarium specimen segmentation. Multimed Tools Appl 81, 28689–28707 (2022). https://doi.org/10.1007/s11042-022-12935-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12935-8

Keywords

Search

Navigation