Skip to main content
Springer Nature Link
Log in

Hybrid deep learning model for multi biotic lesions detection in solanum lycopersicum leaves

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Farmers are concerned about the automatic detection of lesions and pests that threaten tomato plants. Traditional computer vision and pattern recognition technologies have limits when it comes to tackling such difficult problems. However, deep learning had gained popularity in recent years, particularly for the detection and recognition of biotic stress in diseased leaf photos of plants with varying lighting conditions, complicated backdrops, and background noise. In this paper, a system was provided that uses several “Convolutional Neural Networks (CNN)” for the automatic recognition and identification of multi-biotic tomato leaf lesions collected from PlantVillage. For instance segmentation, a “Mask R-CNN” network is used in the first phase; for semantic segmentation a Hybrid Deep Segmentation Convolutional Neural Network Model (Hybrid-DSCNN) model is compared with U-Net and Seg-Net in the second phase, and a CNN model is used for classification in the third stage. The two backbone feature extractors were employed in the Mask R-CNN network, and the ResNet50 displays average precision of 73.00% in the instance segmentation test, which is superior to other models. For the segmentation and classification tasks, the Hybrid-DSCNN 2Layer-Convo-USN has achieved an accuracy of 98.25% which is better than the pre-trained models. The precision of the proposed Hybrid-DSCNN 2Layer-Convo-USN is 95.7% and of U-Net is 94.9%. The results are positive, indicating that the entire system may be implemented in any platform that can be used in the real world.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Algorithm 1
Algorithm 2
Algorithm 3
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Data availability

Data and source codes are available from the authors upon reasonable request.

References

  1. Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39:2481–2495

    Article  Google Scholar 

  2. Barbedo JGA (2016) A review on the main challenges in automatic plant disease identification based on visible range images. Biosyst Eng 144:52–60

    Article  Google Scholar 

  3. Barbedo JGA (2018) Impact of dataset size and variety on the effectiveness of deep learning and transfer learning for plant disease classification. Comput Electron Agric 153:46–53

    Article  Google Scholar 

  4. Bej G, Dey T, Majumdar S, Pal A, Akuli A, Sutradhar T, Kotwaliwale N (2021) Comparison of Different Color Models for Priority Based Color Matching of Plant Parts Used in DUS Testing. In: Proceedings of International Conference on Computational Intelligence, Data Science and Cloud Computing: IEM-ICDC, vol 62, p 471.

  5. Bendjillali RI, Beladgham M, Merit K (2019) Face Recognition Based on DWT Feature for CNN. In: Proceedings of the 9th International Conference on Information Systems and Technologies, pp 1-5

  6. Boulent J, Foucher S, Theau J, St-Charles PL (2019) Convolutional neural networks for the automatic identification of plant diseases. Front Plant Sci 10:941

    Article  Google Scholar 

  7. Chen J, Chen J, Zhang D, Sun Y, Nanehkaran YA (2020) Using deep transfer learning for image-based plant disease identification. Comput Electron Agric 173:105393

    Article  Google Scholar 

  8. De Boer PT, Kroese DP, Mannor S, Rubinstein RY (2005) A Tutorial on the Cross-Entropy Method. Ann Oper Res 134:19–67

    Article  MathSciNet  Google Scholar 

  9. Dutta A, Zisserman A(2019) The VIA annotation software for images, audio and video. In: Proceedings of the 27th ACM International Conference on Multimedia MM ‘19.New York

  10. Esgario JGM, Krohling RA, Ventura JA (2020) Deep learning for classification and severity estimation of coffee leaf biotic stress. Comput Electron Agric 169:105162

    Article  Google Scholar 

  11. Fernández CI, Leblon B, Haddadi A, Wang K, Wang J (2020) Potato late blight detection at the leaf and canopy levels based in the red and red-edge spectral regions. Remote Sens 12:1292

    Article  Google Scholar 

  12. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press. http://www.deeplearningbook.org. Accessed 20 Feb  2022

  13. He K, Gkioxari G, Dollar P, Girshick R(2017) Mask R-CNN. arXiv e-prints, arXiv:1703.06870

  14. Hernandez S, Lopez JL (2020) Uncertainty quantification for plant disease detection using Bayesian deep learning. Appl Soft Comput 96:106597

    Article  Google Scholar 

  15. Hughes DP, Salathe M (2015) An open access repository of images on plant health to enable the development of mobile disease diagnostics. arXiv 2015, arXiv:1511.08060

  16. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on Machine Learning, Lille.

  17. Johannes A, Picon A, Alvarez-Gila A, Echazarra J, Rodriguez-Vaamonde S, Navajas AD, Ortiz-Barredo A (2017) Automatic plant disease diagnosis using mobile capture devices, applied on a wheat use case. Comput Electron Agric 138:200–209

    Article  Google Scholar 

  18. Kaur P, Gautam V (2021) Research patterns and trends in classification of biotic and abiotic stress in plant leaf. Mate Today: Proc 45:4377–4382

    Google Scholar 

  19. Kaur P, Gautam V (2021) Plant Biotic Disease Identification and Classification Based on Leaf Image: A Review. In: Proceedings of 3rd International Conference on Computing Informatics and Networks: ICCIN 2020, Springer Singapore, pp 597-610.

  20. Kaur P, Harnal S, Tiwari R, Upadhyay S, Bhatia S, Mashat A, Alabdali AM (2022) Recognition of Leaf Disease Using Hybrid Convolutional Neural Network by Applying Feature Reduction. Sensors 22:575

    Article  Google Scholar 

  21. Kiptoo GJ (2019). Morphological traits associated with anthracnose (Colletotrichum lindemuthianum) resistance in selected common bean (Phaseolus vulgaris L.) genotypes.

  22. Kukreja, V., Kumar, D., Bansal, A. and Solanki, V., 2022. Recognizing wheat aphid disease using a novel parallel real-time technique based on mask scoring RCNN. In 2022 2nd International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE) (pp. 1372–1377). IEEE.

  23. Li Z, Guo R, Li M, Chen Y, Li G (2020) A review of computer vision technologies for plant phenotyping. Comput Electron Agric 176:105672

    Article  Google Scholar 

  24. Li K, Lin J, Liu J, Zhao Y (2020) Using deep learning for Image-Based different degrees of ginkgo leaf disease classification. Information 11:95

    Article  Google Scholar 

  25. Lu J, Hu J, Zhao G, Mei F, Zhang C (2017) An in-field automatic wheat diseasediagnosis system. Comput Electron Agric 142:369–379

    Article  Google Scholar 

  26. Ma J, Du K, Zheng F, Zhang L, Gong Z, Sun Z (2018) A recognition method forcucumber diseases using leaf symptom images based on deep convolutional neural network. Comput Electron Agric 154:18–24

    Article  Google Scholar 

  27. Manso GL, Knidel H, Krohling RA, Ventura JA (2019) A smartphone application to detection and classification of coffee leaf miner and coffee leaf rust. arXiv e-prints, arXiv:1904.00742

  28. Meyer F (1994) Topographic distance and watershed lines. Signal Process 38:113–125

    Article  Google Scholar 

  29. Minervini M, Fischbach A, Scharr H, Tsaftaris SA (2016) Finely-grained annotated datasets for image-based plant phenotyping. Pattern Recogn Lett 81:80–89

    Article  Google Scholar 

  30. Mishra AM, Harnal S, Mohiuddin K, GautamV NOA, Goyal N, Singh A (2022) A Deep Learning-Based Novel Approach for Weed Growth Estimation. Intell Autom Soft Comput 31:1157–1172

    Article  Google Scholar 

  31. Morris DD (2018) A pyramid CNN for dense-leaves segmentation. arXiv e-prints, arXiv:1804.01646

  32. Pizer SM et al (1987) Adaptive histogram equalization and its variations. Comput Vis, Graph, Image Proc 39:355–368

    Article  Google Scholar 

  33. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. arXiv e-prints, arXiv:1505.04597

  34. Santos TT, De Souza LL, Dos Santos AA, Avila S (2020) Grape detection, segmentation, and tracking using deep neural networks and three-dimensional association. Comput Electron Agric 170:105247

    Article  Google Scholar 

  35. Sharma P, Berwal YPS, Ghai W (2019) Performance analysis of deep learning CNNmodels for disease detection in plants using image segmentation. Inf Process Agric 7:566–574

    Google Scholar 

  36. Tavakoli H, Alirezazadeh P, Hedayatipour A, Nasib AB, Landwehr N (2021) Leaf image-based classification of some common bean cultivars using discriminative convolutional neural networks. Comput Electron Agric 181:105935

    Article  Google Scholar 

  37. Turkoglu M, Hanbay D, Sengur A (2019) Multi-model LSTM-based convolutional neural networks for detection of apple diseases and pests. J Ambient Intell Humaniz Comput, pp 1-11

  38. Uguz S, Uysal N (2021) Classification of olive leaf diseases using deep convolutional neural networks. Neural Comput Applic 33:4133–4149

    Article  Google Scholar 

  39. Wang C, Du P, Wu H, Li J, Zhao C, Zhu H (2021) A cucumber leaf disease severity classification method based on the fusion of DeepLabV3+ and U-Net. Comput Electron Agric 189:106373

    Article  Google Scholar 

  40. Xu L, Li Y, Sun Y, Song L, Jin S (2018) Leaf instance segmentation and counting based on deep object detection and segmentation networks. In: 2018 Joint 10th International Conference on Soft Computing and Intelligent Systems (SCIS) and 19th International Symposium on Advanced Intelligent Systems (ISIS), pp. 180–185

  41. Zhang S, Zhang C (2023) Modified U-Net for plant diseased leaf image segmentation. Comput Electron Agric 204:107511

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Chitkara University Institute of Engineering and Technology, Chitkara University, Punjab, India

    Prabhjot Kaur, Shilpi Harnal & Vinay Gautam

  2. School of Computer Science Engineering and Technology, Bennett University, Greater Noida, India

    Mukund Pratap Singh

  3. Department of Computer Science & Engineering, Rashtrakavi Ramdhari Singh Dinkar College of Engineering, Begusarai, India

    Santar Pal Singh

Authors
  1. Prabhjot Kaur
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Shilpi Harnal
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Vinay Gautam
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Mukund Pratap Singh
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Santar Pal Singh
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Santar Pal Singh.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kaur, P., Harnal, S., Gautam, V. et al. Hybrid deep learning model for multi biotic lesions detection in solanum lycopersicum leaves. Multimed Tools Appl 83, 7847–7871 (2024). https://doi.org/10.1007/s11042-023-15940-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15940-7

Keywords