Abstract
Herbarium specimens form a physical database of plant biodiversity. They are frequently used to study species diversity in different geographic regions or their evolution over time. These specimens have been digitized in bulk and made accessible to the public over the past few decades. These digitization efforts open up new research opportunities for automated processing and analysis. However, few publicly available labeled datasets for herbarium specimens exist. This work introduces a novel instance segmentation dataset of 250 digitized herbarium specimens from a diverse selection of herbaria. We experimented with several segmentation approaches on this dataset and discuss their strengths and limitations. For binary plant segmentation, U-Net and UNet++ achieved IoUs of 0.950 and 0.951, respectively. Popular instance segmentation models could accurately detect common herbaria objects with mask APs between 78.3 and 84.1 but typically struggled with the plant class. Only Mask2Former showed promising results on the plant class, achieving a mask AP of 77.0. Because most herbarium sheets contain a single specimen, the problem was also reformulated as a panoptic segmentation task, treating the plant class as a semantic class. In this context, a combination of YOLOv8 and UNet++ outperformed the Mask2Former model, achieving a higher IoU for the plant class and a higher mask AP for the non-plant objects. The dataset and code are available at: https://github.com/kymillev/herbarium-segmentation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abdelaziz, B.: Walid: a deep learning-based approach for detecting plant organs from digitized herbarium specimen images. Eco. Inform. 69, 101590 (2022). https://doi.org/10.1016/j.ecoinf.2022.101590. https://www.sciencedirect.com/science/article/pii/S1574954122000395
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 833–851. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_49
Cheng, B., Misra, I., Schwing, A.G., Kirillov, A., Girdhar, R.: Masked-attention mask transformer for universal image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1290–1299 (2022)
Dillen, M., et al.: A benchmark dataset of herbarium specimen images with label data. Biodiversity Data J. (7) (2019)
Gaikwad, J., Triki, A., Bouaziz, B.: Measuring morphological functional leaf traits from digitized herbarium specimens using Traitex software. Biodiversity Inf. Sci. Standards 3, e37091 (2019). https://doi.org/10.3897/biss.3.37091
Goëau, H., Bonnet, P., Joly, A.: Overview of lifeCLEF plant identification task 2020. In: CLEF 2020-Conference and Labs of the Evaluation Forum, vol. 2696 (2020)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)
Hussein, B.R., Malik, O.A., Ong, W.-H., Slik, J.W.F.: Semantic segmentation of herbarium specimens using deep learning techniques. In: Alfred, R., Lim, Y., Haviluddin, H., On, C.K. (eds.) Computational Science and Technology. LNEE, vol. 603, pp. 321–330. Springer, Singapore (2020). https://doi.org/10.1007/978-981-15-0058-9_31
Jain, J., Li, J., Chiu, M.T., Hassani, A., Orlov, N., Shi, H.: OneFormer: one transformer to rule universal image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2989–2998 (2023)
Jocher, G., Chaurasia, A., Qiu, J.: YOLO by Ultralytics (2023). https://github.com/ultralytics/ultralytics
Kirillov, A., He, K., Girshick, R., Rother, C., Dollár, P.: Panoptic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9404–9413 (2019)
Kirillov, A., et al.: Segment anything. arXiv preprint arXiv:2304.02643 (2023)
Kirillov, A., Wu, Y., He, K., Girshick, R.: PointRend: image segmentation as rendering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9799–9808 (2020)
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Milleville, K., Thirukokaranam Chandrasekar, K.K., Verstockt, S.: Automatic extraction of specimens from multi-specimen herbaria. ACM J. Comput. Cultural Heritage 16(1), 1–15 (2023)
Owen, D., et al.: Towards a scientific workflow featuring natural language processing for the digitisation of natural history collections. Res. Ideas Outcomes 6, e55789 (2020). https://doi.org/10.3897/rio.6.e55789
Pearson, K.D., et al.: Machine learning using digitized herbarium specimens to advance phenological research. BioScience 70(7), 610–620 (2020). https://doi.org/10.1093/biosci/biaa044
Pohlen, T., Hermans, A., Mathias, M., Leibe, B.: Full-resolution residual networks for semantic segmentation in street scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4151–4160 (2017)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Tan, M., Le, Q.: EfficientNet: rethinking model scaling for convolutional neural networks. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 97, pp. 6105–6114. PMLR (2019). https://proceedings.mlr.press/v97/tan19a.html
Thiers, B.M.: The world’s herbaria 2021: a summary report based on data from index herbariorum (2022). https://sweetgum.nybg.org/science/wp-content/uploads/2022/02/The_Worlds_Herbaria_Jan_2022.pdf
Thiers, B.M., Tulig, M.C., Watson, K.A.: Digitization of the New York botanical garden herbarium. Brittonia 68, 324–333 (2016)
Triki, A., Bouaziz, B., Gaikwad, J., Mahdi, W.: Deep leaf: mask R-CNN based leaf detection and segmentation from digitized herbarium specimen images. Pattern Recogn. Lett. 150, 76–83 (2021)
Wada, K.: Labelme: Image Polygonal Annotation with Python. https://github.com/wkentaro/labelme
White, A.E., Dikow, R.B., Baugh, M., Jenkins, A., Frandsen, P.B.: Generating segmentation masks of herbarium specimens and a data set for training segmentation models using deep learning. Appl. Plant Sci. 8(6), e11352 (2020)
Wilson, R.J., et al.: Applying computer vision to digitised natural history collections for climate change research: temperature-size responses in British butterflies. Methods Ecol. Evol. 14(2), 372–384 (2023)
Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., Girshick, R.: Detectron2 (2019). https://github.com/facebookresearch/detectron2
Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J.: UNet++: a nested U-Net architecture for medical image segmentation. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 3–11. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_1
Acknowledgements
The research activities described in this paper were funded by Ghent University, Imec, and the DiSSCo Flanders project.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Milleville, K., Chandrasekar, K.K.T., Van de Weghe, N., Verstockt, S. (2023). Evaluating Segmentation Approaches on Digitized Herbarium Specimens. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2023. Lecture Notes in Computer Science, vol 14362. Springer, Cham. https://doi.org/10.1007/978-3-031-47966-3_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-47966-3_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47965-6
Online ISBN: 978-3-031-47966-3
eBook Packages: Computer ScienceComputer Science (R0)