Machine vision-based automatic disease symptom detection of onion downy mildew

https://doi.org/10.1016/j.compag.2019.105099Get rights and content

Highlights

  • An image-based field monitoring system for automatically crop monitoring is developed.

  • Onion disease symptom was detected automatically using deep neural networks.

  • The model was trained using the weakly supervised learning method.

  • The mAP at IOU criteria 0.5 was the highest in all models from 74.1 to 87.2.

  • The developed system is possible to save time and cost in field crop cultivation.

Abstract

The effective crop management is major issue in recent agriculture because the cultivation area per farmer is increasing consistently while the aging-related reductions in the labor force. To manage crop cultivation effectively, it needs automatic monitoring in farmland. This paper presents an image-based field monitoring system for automatically crop monitoring and consists of constructing field monitoring system for periodic capturing of onion field images, training the deep neural network model for detecting the disease symptom, and evaluating performance of the developed system. The field monitoring system was composed of a PTZ camera, a motor system, wireless transceiver, and image logging module. The deep learning model was trained based on weakly supervised learning method that can classify and localize objects only with image-level annotation. It is effective to recognize crop disease symptom which has ambiguous boundary. The model was trained using captured onion images using the filed monitoring system, and 6 classes including the disease symptom were classified. The detected disease symptom was localized from background through thresholding of the class activation map. The 60% of maximum value in class activation map was determined as an Optimal threshold for disease symptom localization. Identification performance of disease symptom was evaluated using mAP metric by IoU. The results show that the mAP at IoU criteria 0.5, which should have over 50% overlap, was the highest in all models from 74.1 to 87.2. The results showed that the developed field monitoring system could automatically detect onion disease symptoms in real-time.

Introduction

Onion is a one of the major functional foods for health. The global onion cultivation area increased by 62.96% from 2 million ha in 2000 to 5.4 million ha in 2016, while production increased from 60.5 million tons to 89.8 million tons during the same period (Hanci, 2018). As the demand for onion increases, large-scale cultivation areas should be managed efficiently. However, the management of these areas has suffered a great setback because of crop diseases due to global warming and because of aging-related reductions in the labor force. In particular, more than 50% of the yield loss is caused by disease damage (Harvey et al., 2014), and for this reason, onion growth and disease monitoring is significant for preventing disease damage and minimizing yield loss (Lu et al., 2017). Most disease monitoring for onion cultivation was conducted manually by visual observation; this approach is not only inefficient because the area per worker is too large but also error-prone because the approach depends on skill and health of the workers (Bock et al., 2010). Therefore, the development of an image-based, automated disease observation technology is needed to replace the conventional method (Lee et al., 2016).

Recently, computer vision systems with image processing technology have been highly developed (Camargo and Smith, 2009), and machine learning approaches have enabled this technology to automatically recognize various objects in a manner similar to the human eye. In particular, deep-learning, which is described as hierarchical learning with a deep neural network layer (LeCun et al., 2015), is the most effective technique in research overall and has shown rapid progress for various intelligence tasks such as visual recognition (Krizhevsky et al., 2012, Simonyan and Zisserman, 2015), image captioning (Zhu et al., 2018), multi-image cued story generation (Kim et al., 2018), medical applications (Kourou et al., 2015), autonomous driving (Chen et al., 2018), and other similarly complex analyses with big data. It also has been applied in the field of agriculture for identification of crop diseases. Crop disease identification with deep-learning has several advantages: to separate the disease symptom from complex image backgrounds through trainable feature learning, to identify multiple instances simultaneously, and possibly to employ low-cost field systems with robust crop monitoring. In this context, the approach of various deep-learning techniques was challenged regarding the automatic diagnosis. Lu et al. (2017) exploited deep multiple instance learning to detect wheat diseases using the in-field wheat disease dataset 2017 (WDD2017) collected by mobile cameras, and the results showed an accuracy of greater than 95%. Ferentinos (2018) studied automatic recognition of plant diseases on leaves using various types of deep learning models and compared their accuracy. An openly available database was used to train the model, and the best performance had a success rate of approximately 99%. Mohanty et al. (2016) also conducted plant disease classification based on a deep convolutional neural network, and they reported that the trained model could identify 26 diseases. In addition, more than 40 studies have already been conducted employing deep learning (Kamilaris et al., 2017) for purposes such as insect detection in stored grain (Shen et al., 2018), identification of interline weeds in unmanned aerial vehicle (UAV) images (Bah et al., 2018), and fruit localization and counting (Chen et al., 2017, Rahnemoonfar and Sheppard, 2017, Sa et al., 2016). Furthermore, several big data analysis practices in agriculture and other types of research have been challenged recently to improve performance or to develop new applications. Generally, previous studies have shown high detection performance. However, in most paper, the image data were collected by capturing specific area (diseased region) or using open database which focused on target (Ferentinos, 2018). The real-time images captured in the field might be focused on other objects not only crops area such as weed (Dyrmann et al., 2017), lawn (Armstrong, 2017), or the other part in cultivation area. So, it is difficult to apply the model trained by intended images to field monitoring system for monitoring unspecified scenes continuously. In addition, it was costly to create training dataset, because it is hard to accurately diagnose most crop diseases without annotation of disease presence and region in the images by a pathologist or crop cultivation expert. Although a weakly supervised-based learning approach that can classify and localize objects with image-level annotation was used to diagnose crop disease (Lu et al., 2017), the diseased area was approximately localized without contour verification due to the ambiguous boundary. For this reason, the deep learning-based agriculture applications to diagnose disease have trouble to apply to the field as unmanned monitoring. To overcome this problem, a low-cost annotation-based approach that can be trained using real-time images captured in field is needed.

Real-time monitoring of the suspected symptom can be evaluated onsite by image-based deep-learning easier than diagnosis and can be applied on unmanned alarm systems for agriculture disease forecasting; this system can reduce the labor force and manage crops efficiently. Conventionally, monitoring of the suspected symptom was conducted by a farmer or an educated forecaster. This method is insufficient and inaccurate for monitoring a wide area with the same labor force (Chou et al., 2019) because it is dependent on the person’s ability (skill, tiredness, etc.) and the environmental conditions (weather, obstacles, etc.); this situation makes it difficult to ensure consistent and accurate monitoring, and it raises the concern about the spread of disease by human.

In this study, to reduce farmers’ burden of disease forecasting and minimizing yield loss by disease infection through early detection of suspected disease symptom, an automatic, consistent, and image-based disease monitoring system available onsite was developed. The purpose of the monitoring system is to collect consistently image data of onion cultivation by machine vision, to identify the disease symptom described by the infected portion of crop measurable using the deep neural network model, and to evaluate the system performance. Our work has advantages: (1) it is a fully automated system that includes large-scale image capturing in real-time for unmanned onion field monitoring and disease warning, and (2) a weakly supervised learning approach with localization using an optimal threshold was used, which can better detect the ambiguous boundary between diseased and healthy crop areas in an onion field.

Section snippets

Field monitoring system

The field monitoring system for monitoring onion growth and the disease symptom, as shown in Fig. 1, was developed to enable periodic capturing of onion field images in real time. The system consists of a pan, tilt, zoom (PTZ) camera (HDWC-S322MIR, Honeywell, USA) capable of high-resolution scanning and a motor system with a linear guide for controlling the vertical axis according to crop elevation. In addition, a limit sensor was installed for restricting vertical movement to prevent damage by

Image data collection

The results of the field monitoring system development and captured sample images are shown in Fig. 5. The collected images from the system were consistently transmitted to the image logging module, and then the captured large-scale cultivation image was cropped to 224x224 RGB images and stored normally. Images were automatically taken at regular intervals. Image collection was conducted under various zoom conditions to determine the zoom value suitable for disease monitoring. The zoom value

Conclusions

This study was conducted to develop an onsite real-time automatic disease monitoring system for early detection of suspected disease symptoms. Our research was focused on full automation of the process of real-time image capturing through disease symptom detection for unmanned yield forecasting. Deep learning-based approaches for disease detection have been proposed, and their results show that diseases were detected automatically with high accuracy. These studies were focused on disease

Declaration of Competing Interest

The authors declared that there is no conflict of interest.

Acknowledgments

This work was supported by research fund of Chungnam National University (2018-0608-01)

References (41)

  • Y. Shen et al.

    Detection of stored-grain insects using deep learning

    Comput. Electron. Agric.

    (2018)
  • A. Tealab et al.

    Forecasting of nonlinear time series using ANN

    Futur. Comput. Informatics J.

    (2017)
  • R.P. Thakur et al.

    Downy mildews of India

    Crop Prot.

    (2002)
  • Y. Zhong et al.

    Computational intelligence in optical remote sensing image processing

    Appl. Soft Comput. J.

    (2018)
  • X. Zhu et al.

    Image captioning with triple-attention and stack parallel LSTM

    Neurocomputing

    (2018)
  • G.N. Agrios

    Plant Pathology

    (2005)
  • Armstrong, C., 2017. Using Imagery of Lawns to Estimate Lawn Care Need. Tech. Discl....
  • Bah, M.D., Hafiane, A., Canals, R., 2018. Deep Learning with unsupervised data labeling for weeds detection on UAV...
  • C.H. Bock et al.

    Plant disease severity estimated visually, by digital photography and image analysis, and by hyperspectral imaging

    CRC. Crit. Rev. Plant Sci.

    (2010)
  • V. Buloviene et al.

    Effect of environmental conditions and inocolum concentration on sporulation of Peronospora destructor

    Agron. Res.

    (2006)
  • Cited by (50)

    • Label-efficient learning in agriculture: A comprehensive review

      2023, Computers and Electronics in Agriculture
    • Detection of powdery mildew on strawberry leaves based on DAC-YOLOv4 model

      2022, Computers and Electronics in Agriculture
      Citation Excerpt :

      In the later stage, PM can infect the fruit, reduce the quality of strawberries, and cause serious economic loss of farmers (Jacob et al., 2008). Thus, controlling disease progression in the early stages of PM is essential (Carisse et al., 2013; Kim et al., 2020; Bischoff et al., 2021). Early-onset indications of PM are typically described as when white spots are observed on the leaf.

    • An improved YOLOv5-based vegetable disease detection method

      2022, Computers and Electronics in Agriculture
      Citation Excerpt :

      In the past five years, weed identification (Su et al., 2021; Jiang et al., 2020) and disease detection of plants (Bi et al., 2020; Jiang et al., 2019) using deep learning techniques by convolutional neural networks (CNNs) have been widely used. Unlike traditional machine learning-based models that manually select features, CNNs automatically extract advanced and stable features through an end-to-end pipeline, thus significantly improving the utility of plant leaf detection (Fu et al., 2021; Kim et al., 2020). However, most of the CNN models for vegetable disease detection are implemented on images with simple backgrounds, which limits their application in practical production.

    View all citing articles on Scopus
    View full text