Abstract
Herbarium scans are valuable raw data for studying how plants adapt to climate change and respond to various factors. Characterization of plant traits from these images is important for investigating such questions, thereby supporting plant taxonomy and biodiversity description. However, processing these images for meaningful data extraction is challenging due to scale variance, complex backgrounds that contain annotations, and the variability in specimen color, shape, and orientation of specimens. In addition, the plant organs often appear compressed, deformed, or damaged, with overlapping occurrences that are common in scans. Traditional organ recognition techniques, while adept at segmenting discernible plant characteristics, are limited in herbarium scanning applications. Two automated methods for plant organ identification have been previously reported. However, they show limited effectiveness, especially for small organs. In this study we introduce YOLOv7-ag model, which is a novel model based on the YOLOv7 that incorporates an attention-gate mechanism, which enhances the detection of plant organs, including stems, leaves, flowers, fruits, and seeds. YOLOv7-ag significantly outperforms previous state of the art as well as the original YOLOv7 and YOLOv8 models with a precision and recall rate of 99.2% and 98.0%, respectively, particularly in identifying small plant organs.
Our code is available at https://github.com/IA-E-Col/YOLOv7-ag
H. Ariouat and Y. Sklab—Both authors contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
After the backbone network and before the detection layer.
- 2.
The three scales are designed to detect small, medium, and large objects, respectively. They are represented by feature maps that are extracted at different depths of the neural network, thus allowing for precise detection across a varied range of object sizes.
- 3.
Ground truth represents the desired bounding box as output of the object detection algorithm.
References
Ariouat, H., et al.: Extracting masks from herbarium specimen images based on object detection and image segmentation techniques. Biodiv. Inf. Sci. Stand. 7, e112161 (2023)
Sahraoui, M., Sklab, Y., Pignal, M., Lebbe, R.V., Guigue, V.: Leveraging Multimodality for Biodiversity Data: exploring joint representations of species descriptions and specimen images using CLIP. Biodivers. Inf. Sci. Stand. 7(2023), e112666 (2023)
Meredith, L.: Roles of natural history collections. Ann. Mo. Bot. Gard. 4(83), 536–545 (1996)
Raven, P.H.: Saving plants, saving ourselves. Plants People Planet 1, 8–13 (2019)
Younis, S., Schmidt, M., Weiland, C., Dressler, S., Seeger, B., Hickler, T.: Detection and annotation of plant organs from digitised herbarium scans using deep learning. Biodiv. Data J. 8, e57090 (2020)
Besnard, G., et al.: Herbarium-based science in the twenty-first century. Botany Lett. 165, 323–327 (2018)
Soltis, P.: Digitization of herbaria enables novel research. Am. J. Bot. 104, 1281–1284 (2017)
Abdelaziz, T., Bassem, B., Walid, M.: A deep learning-based approach for detecting plant organs from digitized herbarium specimen images. Eco. Inform. 69, 101590 (2022)
Patrick, W., et al.: Large-scale digitization of herbarium specimens: development and usage of an automated, high-throughput conveyor system. Taxon 67, 165–178 (2018)
Wenli, Z., et al.: Deep-learning-based in-field citrus fruit detection and tracking. Horticult. Res. 9, uhac003 (2022)
Jiang, Y., Li, C.: Convolutional neural networks for image-based high-throughput plant phenotyping: a review. Plant Phenom. 9 (2020)
Borhani, Y., Khoramdel, J., Najafi, E.: A deep learning based approach for automated plant disease classification using vision transformer. Sci. Rep. 12, 11554 (2022)
Ashish, V., et al.: Attention is all you need. CoRR abs/1706.03762 (2017)
Sue Han, L., Chee Seng, C., Simon, J., Paolo, R.: How deep learning extracts and learns leaf features for plant classification. Pattern Recogn. 71, 1–13 (2017)
Mochida, K., et al.: Computer vision-based phenotyping for improvement of plant productivity: a machine learning perspective. GigaScience 8, giy153 (2018)
Shaoqing, R., Kaiming, H., Ross, B.G., Jian, S.: Faster R-CNN: towards real-time object detection with region proposal networks. Computer Vision and Pattern Recognition abs/1506.01497 (2015)
Kaiming, H., Georgia, G., Piotr, D., Ross, B.G.: Mask R-CNN. Computer Vision and Pattern Recognition, vol. abs/1703.06870 (2017)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)
Xiao, Z., Lang, J., Shuai, L., Tingting, Z., Xingang, M.: YOLO-SASE: an improved YOLO algorithm for the small targets detection in complex backgrounds. Computer Vision and Pattern Recognition, vol. abs/2207.02696 (2022)
Chien-Yao, W., Alexey, B., Hong-Yuan Mark, L.: YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. Computer Vision and Pattern Recognition, vol. abs/2207.02696 (2022)
Ozan, O., et al.: Attention U-Net: learning where to look for the pancreas. Computer Vision and Pattern Recognition, vol. abs/1804.03999 (2018)
Lang, P.M., Willems, F., Scheepens, J.F., Burbano, H., Bossdorf, O.: Using herbaria to study global environmental change. New Phytol. 2021, 110–122 (2019)
Zhao, H., Zhang, H., Zhao, Y.: YOLOv7-Sea: object detection of maritime UAV images based on improved YOLOv7. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops (2023)
Zixiao, Z., et al.: ViT-YOLO: transformer-based YOLO for object detection. In: IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2021, Montreal, BC, Canada, 11–17 October 2021
James, C., et al.: From prototype to inference: a pipeline to apply deep learning in sorghum panicle detection. Plant Phenomics 5, 0017 (2023)
Jie, X., et al.: TrichomeYOLO: a neural network for automatic maize trichome counting. Plant Phenom. 5, 0024 (2023)
Zhaoyang, N., Guoqiang, Z., Hui, Y.: A review on the attention mechanism of deep learning. Plant Neurocomput. 452, 48–62 (2021)
Dillon, R., Jordan, K., Jacqueline, H., Ahmad, D.: Real-time flying object detection with YOLOv8. Computer Vision and Pattern Recognition (2023)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Ariouat, H. et al. (2024). Enhancing YOLOv7 for Plant Organs Detection Using Attention-Gate Mechanism. In: Yang, DN., Xie, X., Tseng, V.S., Pei, J., Huang, JW., Lin, J.CW. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2024. Lecture Notes in Computer Science(), vol 14646. Springer, Singapore. https://doi.org/10.1007/978-981-97-2253-2_18
Download citation
DOI: https://doi.org/10.1007/978-981-97-2253-2_18
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-2252-5
Online ISBN: 978-981-97-2253-2
eBook Packages: Computer ScienceComputer Science (R0)