Skip to main content

Advertisement

SpringerLink
Log in

Fruits diseases classification: exploiting a hierarchical framework for deep features fusion and selection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In agriculture farming business, plant diseases are one of the reasons for the financial deficits around the globe. It is the fundamental factor, as it causes significant abatement in both capacity and quality of the growing crops. In plants, fruits are amongst the major sources of nutrients, however, there exists a wide range of diseases which adversely affect both quality and production of the fruits. To overcome such predicament, computer vision (CV) based methods are introduced. These methods are quite effective, which not only detect the diseases/infections at the early stages but also assign them a label. In this article, we propose a deep convolutional neural network-based method for the diseases classification of different fruits’ leaves. Initially, the deep features are extracted by utilizing pre-trained deep models including VGG-s and AlexNet, which are later fine-tuned by employing a concept of transfer learning. A multi-level fusion methodology is also proposed, prior to the selection step, based on an entropy-controlled threshold value - calculated by averaging the selected features. The resultant final feature vector is later fed into a host classifier, multi-SVM. Five different diseases are considered for experiments including apple black rot, apple scab, apple rust, cherry powdery mildew, and peach bacterial spots, which are collected from a plant village dataset. Classification results clearly reveal the improved performance of proposed method in terms of sensitivity (97.6%), accuracy (97.8%), precision (97.6%), and G-measure (97.6%).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Abdel-Hamid O, Mohamed A-R, Jiang H, Penn G (2012) Applying convolutional neural networks concepts to hybrid NN-HMM model for speech recognition. In: 2012 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, pp 4277-4280

  2. Adeel A, Khan MA, Akram T, Sharif A, Yasmin M, Saba T, Javed K (2020) Entropy-controlled deep features selection framework for grape leaf diseases recognition. Expert Systems

  3. Abu-Naser SS, Kashkash KA, Fayyad M (2010) Developing an expert system for plant disease diagnosis. J Artif Intell 3(4):269–276

    Google Scholar 

  4. Ai D, Yang J, Wang Z, Fan J, Ai C, Wang Y (2015) Fast multi-scale feature fusion for ECG heartbeat classification. EURASIP Journal on Advances in Signal Processing 2015(1):46

    Google Scholar 

  5. Amara J, Bouaziz B, Algergawy A (2017) A deep learning-based approach for banana leaf diseases classification. In: BTW (Workshops), pp 79–88

  6. Barbedo JGA (2016) A review on the main challenges in automatic plant disease identification based on visible range images. Biosystems Engineering 144:52–60

    Google Scholar 

  7. Bengio Y, Courville A, Vincent P (2013) Representation learning: A review and new perspectives. IEEE Trans Pat Anal Mach Intel 35(8):1798–1828

    Google Scholar 

  8. Brahimi M, Boukhalfa K, Moussaoui A (2017) Deep learning for tomato diseases Classification and symptoms visualization. Appl Artif Intell 31 (4):299–315

    Google Scholar 

  9. Cai W, Wei Z (2020) PiiGAN: Generative adversarial networks for pluralistic image inpainting. IEEE Access 8:48451–48463

    Google Scholar 

  10. Chaib S, Liu H, Gu Y, Yao H (2017) Deep feature fusion for VHR remote sensing scene classification. IEEE Trans Geosci Remote Sens 55 (8):4775–4784

    Google Scholar 

  11. Chatfield K, Simonyan K, Vedaldi A, Zisserman A (2014) Return of the devil in the details: Delving deep into convolutional nets. arXiv:1405.3531

  12. Chen L, Yang M (2017) Semi-supervised dictionary learning with label propagation for image classification. Computational Visual Media 3(1):83–94

    MATH  Google Scholar 

  13. Chen X, Kundu K, Zhu Y, Ma H, Fidler S, Urtasun R (2017) 3D object proposals using stereo imagery for accurate object class detection. IEEE transactions on pattern analysis and machine intelligence

  14. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In: IEEE Conference on computer vision and pattern recognition, 2009. CVPR 2009, IEEE, pp 248–255

  15. Dubey SR, Jalal AS (2012) Detection and classification of apple fruit diseases using complete local binary patterns. In: 2012 Third international conference on computer and communication technology (ICCCT), IEEE, pp 346–351

  16. Dubey SR, Jalal AS (2016) Apple disease classification using color, texture and shape features from images. SIViP 10(5):819–826

    Google Scholar 

  17. Duan Y, Liu F, Jiao L, Zhao P, Zhang L (2017) SAR Image Segmentation based on convolutional-wavelet neural network and markov random field. Pattern Recogn 64:255–267

    Google Scholar 

  18. El Faouzi N-E, Leung H, Kurian A (2011) Data fusion in intelligent transportation systems Progress and challenges–A survey. Information Fusion 12(1):4–10

    Google Scholar 

  19. Elangovan K, Nalini S (2017) Plant disease classification using image segmentation and SVM techniques. Int J Comput Intell Res 13(7):1821–1828

    Google Scholar 

  20. Espinosa JE, Velastin SA, Branch JW (2017) Vehicle detection using alex net and faster R-CNN deep learning models: a comparative study. In: International visual informatics conference. Springer, Cham, pp 3–15

  21. Fan D-P, Cheng M-M, Liu J-J, Gao S-H, Hou Q, Borji A (2018) Salient objects in clutter Bringing salient object detection to the foreground. In: Proceedings of the European conference on computer vision (ECCV), pp 186–202

  22. Fernando B, Fromont E, Tuytelaars T (2014) Mining mid-level features for image classification. Int J Comput Vis 108(3):186–203

    MathSciNet  Google Scholar 

  23. Fuentes A, Yoon S, Kim SC, Park DS (2017) A robust deep-learning-based detector for real-time tomato plant diseases and pests recognition. Sensors 17(9):2022

    Google Scholar 

  24. Fu K, Zhao Q, Gu IY-H, Yang J (2019) Deepside: A general deep framework for salient object detection. Neurocomputing 356:69–82

    Google Scholar 

  25. Gunes H, Piccardi M (2005) Affect recognition from face and body: early fusion vs. late fusion. In: 2005 IEEE International conference on systems, man and cybernetics, vol 4, IEEE, pp 3437–3443

  26. Hautamäki V, Siniscalchi SM, Behravan H, Salerno VM, Kukanov I (2015) Boosting universal speech attributes classification with deep neural network for foreign accent characterization. In Sixteenth annual conference of the international speech communication association

  27. Hedayatnia B, Yazdani M, Nguyen M, Block J, Altintas I (2016) Determining feature extractors for unsupervised learning on satellite images. In: 2016 IEEE international conference on big data (big data), IEEE, pp 2655–2663

  28. Hijazi S, Kumar R, Rowen C (2015) Using convolutional neural networks for image recognition. Cadence Design Systems Inc., San Jose

    Google Scholar 

  29. Hughes D, Salathé M (2015) An open access repository of images on plant health to enable the development of mobile disease diagnostics. arXiv:1511.08060

  30. Iqbal Z, Khan MA, Sharif M, Shah JH, ur Rehman MH, Javed K (2018) An automated detection and classification of citrus plant diseases using image processing techniques: A review. Computers and Electronics in Agriculture 153:12–32

    Google Scholar 

  31. Jeong JH, Kwon S, Hong M-P, Kwak J, Shon T (2019) Adversarial attack-based security vulnerability verification using deep learning library for multimedia video surveillance. Multimedia Tools and Applications, pp 1–15

  32. Kamilaris A, Prenafeta-Boldú FX (2018) Deep learning in agriculture A survey. Computers and Electronics in Agriculture 147:70–90

    Google Scholar 

  33. Khan MA, Sharif M, Javed MY, Akram T, Yasmin M, Saba T (2017) License number plate recognition system using entropy-based features selection approach with SVM IET. Image Processing

  34. Khan MA, Akram T, Sharif M, Awais M, Javed K, Ali H, Tanzila S. (2018) CCDF Automatic System for segmentation and recognition of fruit crops diseases based on correlation coefficient and deep CNN features. Comput Electron Agric 155:220–236

    Google Scholar 

  35. Khan MA, Akram T, Sharif M, Javed MY, Muhammad N, Yasmin M (2018) An implementation of optimized framework for action classification using multilayers neural network on selected fused features. Pattern Anal Applic, pp 1–21

  36. Khan MA, Ikram Ullah Lali M, Sharif M, Javed K, Aurangzeb K, Haider SI, Altamrah AS, Akram T (2019) An optimized method for segmentation and classification of apple diseases based on strong correlation and genetic algorithm based feature selection. IEEE Access 7:46261–46277

    Google Scholar 

  37. Khan MA, Akram T, Sharif M, Javed K, Raza M, Saba T (2020) An automated system for cucumber leaf diseased spot detection and classification using improved saliency method and deep features selection. Multimedia Tools and Applications, pp.1-30

  38. Khan MA, Akram T, Sharif M, Awais M, Javed K, Ali H, Saba T (2018) CCDF: Automatic system for segmentation and recognition of fruit crops diseases based on correlation coefficient and deep CNN features. Computers and electronics in agriculture 155:220–236

    Google Scholar 

  39. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  40. Kumar KPS, Bhavani R (2017) Human activity recognition in egocentric video using PNN SVM, kNN and SVM+ kNN classifiers. Clust Comput, pp 1–10

  41. Lad H, Mehta MA (2017) Feature based object mining and tagging algorithm for digital images. In: Inproceedings of international conference on communication and networks. Springer, Singapore, pp 345–352

  42. Li D, Sun B, Chen X, Yu G, Wang H (2017) Gait recognition of lower limb rehabilitation robot based on support vector machine. In: Proceedings of the 2017 international conference on artificial intelligence, automation and control technologies, ACM, pp 29

  43. Liu B, Zhang Y, He DJ, Li Y (2017) Identification of apple leaf diseases based on deep convolutional neural networks. Symmetry 10(1):11

    Google Scholar 

  44. Liu Y i, Zheng YF One-against-all multi-class SVM classification using reliability measures. In: 2005 IEEE international joint conference on neural networks, 2005. IJCNN’05. Proceedings, vol 2. IEEE, pp 849–854

  45. Liu B, Zhang Y, He DJ, Li Y (2018) Identification of apple leaf diseases based on deep convolutional neural networks. Symmetry 10(1):11

    Google Scholar 

  46. Luus FPS, Salmon BP, Van den Bergh F, Maharaj BTJ (2015) Multiview deep learning for land-use classification. IEEE Geosci Remote Sens Lett 12(12):2448–2452

    Google Scholar 

  47. Mangai UG, Samanta S, Das S, Chowdhury PR (2010) A survey of decision fusion and feature fusion strategies for pattern classification. IETE Technical Review 27(4):293–307

    Google Scholar 

  48. Mohanty SP, Hughes DP, Salathé M (2016) Using deep learning for image-based plant disease detection. Frontiers in Plant Science 7:1419

    Google Scholar 

  49. Neto A, Miranda A, Victorino C, Fantoni I, Zampieri DE, Ferreira JV, Lima DA (2013) Image processing using Pearson’s correlation coefficient: Applications on autonomous robotics. In 2013 13th international conference on autonomous robot systems (Robotica), IEEE, pp 1–6

  50. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering 22(10):1345–1359

    Google Scholar 

  51. Parkhi OM, Vedaldi A, Zisserman A (2015) Deep face recognition. BMVC 1(3):6

    Google Scholar 

  52. Piao Y, Rong Z, Zhang M, Lu H Exploit and Replace: An Asymmetrical Two-Stream Architecture for Versatile Light Field Saliency Detection

  53. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, et al. (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    MathSciNet  Google Scholar 

  54. Salerno V, Rabbeni G (2018) An extreme learning machine approach to effective energy disaggregation. Electronics 7(10):235

    Google Scholar 

  55. Schmidhuber J (2015) Deep learning in neural networks: An overview. Neural Networks 61:85–117

    Google Scholar 

  56. Sharif M, Khan MA, Iqbal Z, Azam MF, Ikram Ullah Lali M, Javed MY (2018) Detection and classification of citrus diseases in agriculture based on optimized weighted segmentation and feature selection. Comput Electron Agric 150:220–234

    Google Scholar 

  57. Sharif M, Khan MA, Faisal M, Yasmin M, Fernandes SL (2018) A Framework for Offline Signature Verification System. Best Features Selection Approach. Pattern Recognition Letters

  58. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  59. Singh M, Singh S, Gupta S (2014) An information fusion based method for liver classification using texture analysis of ultrasound images. Information Fusion 19:91–96

    Google Scholar 

  60. Singh C, Mittal N, Walia E (2014) Complementary feature sets for optimal face recognition. EURASIP Journal on Image and Video Processing 2014 (1):35

    Google Scholar 

  61. Sladojevic S, Arsenovic M, Anderla A, Culibrk D, Stefanovic D (2016) Deep neural networks based recognition of plant diseases by leaf image classification. Computational intelligence and neuroscience 2016

  62. Strange RN, Scott PR (2005) Plant disease: a threat to global food security. Annual Review of Phytopathology 43

  63. Tajbakhsh N, Shin JY, Gurudu SR, Todd Hurst R, Kendall CB, Gotway MB, Liang J (2016) Convolutional neural networks for medical image analysis Full training or fine tuning?. IEEE Transactions on Medical Imaging 35 (5):1299–1312

    Google Scholar 

  64. Tang P, Wang H (2017) Richer feature for image classification with super and sub kernels based on deep convolutional neural network. Computers & Electrical Engineering 62:499–510

    Google Scholar 

  65. Tri NC, Van Hoai T, Duong HN, Trong NT, Van Vinh V, Snasel V (2016) A novel framework based on deep learning and unmanned aerial vehicles to assess the quality of rice fields. In: International conference on advances in information and communication technology. Springer, Cham, pp 84–93

  66. van Doorn J (2014) Analysis of deep convolutional neural network architectures: 9-12

  67. Wang L, Qiao Y, Tang X (2015) Action recognition with trajectory-pooled deep-convolutional descriptors. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4305–4314

  68. Wahba MA, Ashour AS, Napoleon SA, Elnaby MMAbd, Guo Y (2017) Combined empirical mode decomposition and texture features for skin lesion classification using quadratic support vector machine. Health Information Science and Systems 5(1):10

    Google Scholar 

  69. Weiss K, Khoshgoftaar TM, Wang DD (2016) A survey of transfer learning. Journal of Big Data 3(1):9

    Google Scholar 

  70. Xie D, Zhang L, Li B (2017) Deep learning in visual computing and signal processing. Applied Computational Intelligence and Soft Computing 2017

  71. Xu J, Bo T, He H, Man H (2017) Semisupervised feature selection based on relevance and redundancy criteria. IEEE T. Neural. Networ.and Learning Systems 28(9):1974–1984

    MathSciNet  Google Scholar 

  72. Xu G, Zhu X, Fu D, Dong J, Xiao X (2017) Automatic land cover classification of geo-tagged field photos by deep learning. Environmental Modelling & Software 91:127–134

    Google Scholar 

  73. Yang J, Yang J-Y, Zhang D, Lu J-F (2003) Feature fusion: parallel strategy vs. serial strategy. Pattern Recognition 36(6):1369–1381

    MATH  Google Scholar 

  74. Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? In: Advances in neural information processing systems, pp 3320–3328

  75. Zhang A, Wang H, Li S, Cui Y, Liu Z, Yang G, Hu J (2018) Transfer learning with deep recurrent neural networks for remaining useful life estimation. Appl Sci 8(12):2416

    Google Scholar 

  76. Zhao J-X, Liu J-J, Fan D-P, Cao Y, Yang J, Cheng M-M (2019) EGNet: Edge guidance network for salient object detection. In: Proceedings of the IEEE international conference on computer vision, pp 8779–8788

  77. Zhong Y, Zhao M (2020) Research on deep learning in apple leaf disease recognition. Computers and Electronics in Agriculture 168:105146

    Google Scholar 

  78. Zhou W, Newsam S, Li C, Shao Z (2017) Learning low dimensional convolutional neural networks for high-resolution remote sensing image retrieval. Remote Sens 9(5):489

    Google Scholar 

Download references

Acknowledgements

The authors greatly acknowledge COMSATS University Islamabad, Wah Campus, and HITEC University for providing resources through out this project.

Funding

Not applicable

Author information

Authors and Affiliations

  1. Department of Computer Science, HITEC University, Museum Road, Taxila, Pakistan

    Muhammad Attique khan

  2. Department of EE, COMSATS University Islamabad, Wah Cantt, Pakistan

    Tallha Akram

  3. Department of Computer Science, COMSATS University Islamabad, Wah Cantt, Pakistan

    Muhammad Sharif

  4. College of Computer and Information Sciences, Prince Sultan University, Riyadh, Saudi Arabia

    Tanzila Saba

Authors
  1. Muhammad Attique khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tallha Akram
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muhammad Sharif
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tanzila Saba
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muhammad Attique khan.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Ethics approval and consent to participate

“ Not applicable ”

Consent for publication

“ Not applicable ”

Availability of data and material

“Plant Village Dataset”

Authors’ contributions

MAK generate this idea and perform simulation, he is also responsible for ist draft. TA gives technical supports. MS do the prrof reading. TS gives the final shape of the manuscript.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

khan, M.A., Akram, T., Sharif, M. et al. Fruits diseases classification: exploiting a hierarchical framework for deep features fusion and selection. Multimed Tools Appl 79, 25763–25783 (2020). https://doi.org/10.1007/s11042-020-09244-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09244-3

Keywords

Search

Navigation