Abstract
In this study, taking the common diseases in tomato leaves, which are typical crops in southern China, as the research object, a FC-SNDPN (Fully Convolutional – Switchable Normalization Dual Path Networks) -based method for automatic identification and detection of crop leaf diseases is proposed to solve the problem that traditional image identification methods for crop diseases and insect pests heavily rely on artificial feature extraction and have a poor generalization ability for image recognition with a complex background. In order to reduce the influence of the complicated background on the recognition of crops diseases and insect pests image, A full convolutional network (FCN) algorithm based on VGG-16 model is used to segment the target crop image. Then an improved DPN (Dual-Path Networks) model is proposed to improve the ability of feature extraction. SNDPN combines the connection method between Desnet and Resnet layers, forms a neural network by using SN layer, and adaptively optimizes the parameters of the dual-path neural network by switching the normalized layer, which improves the versatility of the network for different types of diseases and insect pests and the training speed of the network. Finally, the identification accuracy of the proposed method of using FCN for foreground segmentation and SNDPN for identification is 97.59% on the augmentation data set, the result proves the effectiveness of our method.
Similar content being viewed by others
References
Alenyà Ribas G, Dellen B, Foix Salmerón S, Torras C (2013) Robotized Plant Probing: Leaf Segmentation Utilizing Time-Of Flight Data. IEEE Robot Automat Mag 20(3):50–59. https://doi.org/10.1109/MRA.2012.2230118
Belasque Jr J, Gasparoto M, Marcassa L G (2008) Detection of Mechanical and Disease Stresses in Citrus Plants By Fluorescence Spectroscopy. Appl Opt 47(11):1911–1926.2 https://doi.org/10.1364/AO.47.001922
Bharate AA, Shirdhonkar MS (2017) A review on plant disease detection using image processing. In: 2017 international conference on intelligent sustainable systems (ICISS). IEEE, pp 103–109. https://doi.org/10.1109/ISS1.2017.8389326
Brahimi M, Boukhalfa K, Moussaoui A (2017) Deep Learning for Tomato Diseases: Classsification and Symptoms Visualization. Appl Artif Intell 31:299–315. https://doi.org/10.1080/08839514.2017.1315516
Bravo C, Moshou D, Oberti R, West J, McCartney A, Bodria L, Ramon H (2004) Foliar disease detection in the field using optical sensor fusion. Agric Eng Int: CIGR J. Retrieved May 20, 2020, from http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.111.6150
Camargo A, Smith J (2009) An image-processing based algorithm to automatically identify plant disease visual symptoms. Biosyst Eng 102(1):9–21. https://doi.org/10.1016/j.biosystemseng.2008.09.030
Chaerle L, Lenk S, Hagenbeek D, Buschmann C, Van Der Straeten D (2007) Multicolor Fluorescence Imaging for Early Detection of the Hypersensitive Reaction to Tobacco Mosaic Virus[J]. Plant Physiol 164(3):253–262. https://doi.org/10.1093/jxb/erl207
Chen Y, Li J, Xiao H et al (2017) Dual path networks [J]. Retrieved March 10, 2020, from http://papers.nips.cc/paper/7033-dual-path-networks
Durmu H, Güne EO, Kirci M (2017) Disease Detection on the Leaves of the Tomato Plants by Using Deep Learning[C]//Agro-Geoinformatics,2017 6th International Conference on, IEEE. https://doi.org/10.1109/Agro-Geoinformatics.2017.8047016
Dyrmann M, Karstoft H, Midtiby HS (2016) Plant Species Classification using Deep Convolutional Neural Network. Biosystem Eng. 151:72–80. https://doi.org/10.1016/j.biosystemseng.2016.08.024
Fan DP, Lin Z, Zhang Z, Zhu M, Cheng MM (2020) Rethinking RGB-D Salient Object Detection: Models, Data Sets, And Large-Scale Benchmarks. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2020.2996406
Ferentinos KP (2017) Deep Learning Models for Plant Disease Detection and Diagnosis. Comput Electron Agric 145:311–318. https://doi.org/10.1016/j.compag.2018.01.009
Fu K, Fan DP, Ji GP, Zhao Q (2020) Jl-dcf: joint learning and densely-cooperative fusion framework for Rgb-D salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3052–3062. Retrieved August 6, 2020, from https://arxiv.org/abs/2004.08515
Fu K, Zhao Q, Gu IYH, Yang J (2019) Deepside: A General Deep Framework for Salient Object Detection. Neurocomputing 356:69–82. https://doi.org/10.1016/j.neucom.2019.04.062
Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587. Retrieved July 13, 2020, from http://openaccess.thecvf.com/content_cvpr_2014/html/Girshick_Rich_Feature_Hierarchies_2014_CVPR_paper.html
Grinblat GL, Uzal LC, Larese MG, Granitto PM (2016) Deep Learning for Plant Identification using Vein Morphological Patterns. Comput Electron Agric 127:418–424. https://doi.org/10.1016/j.compag.2016.07.003
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778. Retrieved July 28, 2019, from http://openaccess.thecvf.com/content_cvpr_2016/html/He_Deep_Residual_Learning_CVPR_2016_paper.html
He K, Sun J (2015) Convolutional neural networks at constrained time cost. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5353–5360. Retrieved June 10, 2019, from https://www.cv-foundation.org/openaccess/content_cvpr_2015/html/He_Convolutional_Neural_Networks_2015_CVPR_paper.html
Houby E M F EI (2018) A Survey on Applying Machine Learning Techniques for Management Of Diseases[J]. Appl Biomed 16(3):165–174 https://doi.org/10.1016/j.jab.2018.01.002
Huang G, Liu Z, Laurens VDM et al (2016) Densely connected convolutional networks [J]. Retrieved February 26, 2020, from https://arxiv.org/abs/1608.06993
Hughes D, Salathé M (2015) An open access repository of images on plant health to enable the development of mobile disease diagnostics. arXiv preprint arXiv:1511.08060. https://doi.org/10.48550/arXiv.1511.08060
Keizhevky A, Sutskever L, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105. Retrieved October 19, 2019, from http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networ
Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation [C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition. Retrieved May 25, 2019, from https://www.cv-foundation.org/openaccess/content_cvpr_2015/html/Long_Fully_Convolutional_Networks_2015_CVPR_paper.html
Lu J, Hu J, Zhao G, Mei F, Zhang C (2017) An in-field automatic wheat disease diagnosis system. Comput Electron Agric 142:369–379. https://doi.org/10.1016/j.compag.2017.09.012
Lu X, Wang W, Ma C, Shen J, Shao L, Porikli F (2019) See more, know more: unsupervised video object segmentation with co-attention siamese networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3623–3632. Retrieved April 7, 2020, from https://doi.org/10.1109/CVPR.2019.00374
Lu X, Wang W, Shen J, Tai YW, Crandall DJ, Hoi SC (2020) Learning video object segmentation from unlabeled videos. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8960–8970. Retrieved May 16, 2020, from https://arxiv.org/abs/2003.05020
Lu X, Wang W, Danelljan M, Zhou T, Shen J, Van Gool L (2020) Video object segmentation with episodic graph memory networks. arXiv preprint arXiv:2007.07020. Retrieved May 19, 2020, from https://arxiv.org/abs/2007.07020
Luo P, Ren J, Peng Z (2019) Differentiable learning-to-normalize via switchable normalization. In: International conference on learning representation. Retrieved June 3, 2020, from https://arxiv.org/abs/1806.10779
Mahmoud MA, Guo P, Wang K (2020) Pseudoinverse Learning Autoencoder with DCGAN for Plant Diseases Classification. Multimedia Tools Applic 79(35):26245–26263. https://doi.org/10.1007/s11042-020-09239-0
Mohanty SP, Hughes DP, Salathé M (2016) Using deep learning for image-based plant disease detection. Front Plant Sci 7:1419. https://doi.org/10.3389/fpls.2016.01419
Moshou D, Bravo C, West J et al (2004) Automatic Detection of “Yellow Rust” in Wheat Using Reflectance Measurements and Neural Networks. Comput Electron Agric 44:173–188. https://doi.org/10.1016/j.compag.2004.04.003
Moshou D, Bravo C, Oberti R, West J, Bodria L, McCartney A, Ramon H (2005) Plant Disease Detection Based on Data Fusion of Hyper-Spectral and Multi-Spectral Fluorescence Imaging Using Kohonen Maps. Real-Time lmag 11(2):75–83. https://doi.org/10.1016/j.rti.2005.03.003
Nachtigall LG, Araujo RM, Nachltigall GR (2016) Classification of apple tree disorders using convolution neural networks [C]. In: 2016 IEEE 28th international conference on tools with artificial intelligence (ICTAI). IEEE. https://doi.org/10.1109/ICTAI.2016.0078
Nair V, Hinton GE (2010) Rectified linear units improve restricted Boltzmann Machines [C]. In: Proceedings of the 27th international conference on machine learning (ICML-10). Omnipress, Madison, pp 807–814. Retrieved September 8, 2019, from https://www.cs.toronto.edu/~hinton/absps/reluICML.pdf
Phadikar S, Sil J, Das AK (2013) Rice Diseases Classification using Feature Selection and Rule Generation Techniques. Comp Electron Agric 90:76–85. https://doi.org/10.1016/j.compag.2012.11.001
Powers DM (2020) Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. arXiv preprint arXiv:2010.16061. https://doi.org/10.48550/arXiv.2010.16061
Qin J, Burks TF, Ritenour MA, Bonn WG (2009) Detection of Citrus Canker Using Hyperspectral Reflectance Imaging With Spectral Information Divergence[J]. Food Eng 93(2):183–191. https://doi.org/10.1016/j.jfoodeng.2009.01.014
Ren S, He K, Girshick R et al (2017) Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Network[J]. IEEE Trans Pattern Anal Mach Intell 36(6):1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031
Sa I, Ge Z, Dayoub F, Upcroft B, Perez T, McCool C (2016) Deepfruits: A Fruit Detection System Using Deep Neural Networks. Sensors 16(3):1222. https://doi.org/10.3390/s16081222
Sermanet P, Eigen D, Zhang X, Fergus R, LeCun Y (2013) Localization and detection using convolutional networks. In: Overfeat: integrated recognition. Retrieved October 14, 2019, from https://arxiv.gg363.site/abs/1312.6229
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409. https://doi.org/10.48550/arXiv.1409.1556
Sladojevic S, Arsenovic M, Anderla A, Culibrk D, Stefanovic D (2016) Deep Neural Networks Based Recognition of Plant Diseases by Leaf Image Classification. Comput Intell Neurosci. https://doi.org/10.1155/2016/3289801
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9. https://doi.org/10.1109/CVPR.2015.7298594
Uijlings JR, van de Sande KE, Gevers T, Smeulders AW (2013) Selective Search for Object Recognition. Int J Comput Version 104(2):154–171. https://doi.org/10.1007/s11263-013-0620-5
Wang J, He J, Han Y, Ouyang C, Li D (2013) An Adaptive Thresholding Algorithm of Field Leaf Image. Comput Electron Agric 96:23–39. https://doi.org/10.1016/j.compag.2013.04.014
Wang D, Vinson R, Holmes M, Seibel G, Bechar A, Nof S, Tao Y (2019a) Early detection of tomato spotted wilt virus by hyperspectral imaging and outlier removal auxiliary classifier generative adversarial nets (OR-AC-GAN). Sci Rep 9(1):1–14. https://doi.org/10.3390/plants8110468
Wang W, Lu X, Shen J, Crandall DJ, Shao L (2019b) Zero-shot video object segmentation via attentive graph neural networks. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 9236–9245. https://doi.org/10.48550/arXiv.2001.06807
Wason R (2018) Deep Learning: Evolution and Expansion. Cogn Syst Res 2018(52):701–708. https://doi.org/10.1016/j.cogsys.2018.08.023
Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1492–1500. https://doi.org/10.48550/arXiv.1611.05431
Zhang M, Meng Q (2011) Automatic Citrus Canker Detection from Leaf Images Captured in Field. Pattern Recogn Lett 32(15):2036–2046. https://doi.org/10.1016/j.patrec.2011.08.003
Zhao JX, Liu JJ, Fan DP, Cao Y, Yang J, Cheng MM (2019) EGNet: edge guidance network for salient object detection. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 8779–8788. https://doi.org/10.1109/ICCV.2019.00887
Acknowledgements
The author would like to thank China’s national 948 program for its funding. Meanwhile, the author would also like to thank Yang Xiaobo for his illustrations and experimental assistance, as well as Yi jizheng, a professor at Central South University of Forestry and Technology, for some helpful suggestions. The references are arranged according to the citations in the paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Huang, X., Chen, A., Zhou, G. et al. Tomato Leaf Disease Detection System Based on FC-SNDPN. Multimed Tools Appl 82, 2121–2144 (2023). https://doi.org/10.1007/s11042-021-11790-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-021-11790-3