Intelligent detection of citrus fruit pests using machine vision system and convolutional neural network through transfer learning technique

Highlights

• The detection accuracy of infected fruits with Mediterranean larvae was obtained using transfer learning technique 99.33%.

• Performance comparisons between four CNN pre-trained models for detecting infected fruits were examined.

• The comparison of efficiency between the three optimization algorithms in the models training process was investigated.

• The findings and topics of this article will be useful to help researchers identify pests and diseases.

Abstract

Plant pests and diseases play a significant role in reducing the quality of agricultural products. As one of the most important plant pathogens, pests like Mediterranean fruit fly cause significant damage to crops and thus annually farmers face a lot of loss in their products. Therefore, the use of modern and non-destructive methods such as machine vision systems and deep learning for early detection of pests in agricultural products is of particular importance. In this study, citrus fruit images were taken in three stages: 1) before pest infestation, 2) beginning of fruit infestation, and 3) eight days after the second stage, in natural light conditions (7000–11,000 lux). A total of 1519 images were prepared for all classes. To classify the images, 70% of the images were used for the network training stage, 10% and 20% of the images were used for the validation and testing stages. Four pre-trained CNN models, namely ResNet-50, GoogleNet, VGG-16 and AlexNet as well as the SGDm, RMSProp and Adam optimization algorithms were used to identify and classify healthy fruit and fruit infected with the Mediterranean fly. The results of evaluating the models in the pest outbreak stage showed that the VGG-16 model with the help of SGDm algorithm had the best efficiency with the highest detection accuracy and F1 of 98.33% and 98.36%, respectively. The evaluation of the third stage showed that the AlexNet model with the help of SGDm algorithm had the best result with the highest detection accuracy and F1 of 99.33% and 99.34%, respectively. AlexNet model using SGDm optimization algorithm had the shortest network training time (323 s). The results of this study showed that convolutional neural network method and machine vision system can be effective in controlling and managing pests in orchards and other agricultural products.

Introduction

Nowadays, pest and disease control are crucial steps in reducing crop losses. One of the most destructive pests that affect agricultural products such as citrus fruits is the Mediterranean fruit fly (MFF) [1]. With its high reproduction rate, adaptation to different climatic conditions, lack of natural enemies and the existence of more than 350 species of hosts (fruits and vegetables) for it, this pest has been able to spread rapidly around the world [2,3]. The life cycle of this pest is between 21 and 30 days and can be described in four consecutive stages which consist of 1) the penetration of the female fly's bite into the fruit and laying eggs inside it, 2) conversion of eggs into larva that feed on the fruit, resulting in gradual spoilage due to the penetration of fungi and bacteria through the tunneling created in the fruit, 3) exit of larva from the fruit to pupate and its fall on the soil and 4) passing the maturation stage in the soil and departure from the soil; beginning to mate after five days [4,5]. Successive generations of flies are born after several months due to the availability of hosts. Common methods for controlling this pest in the world are the use of insecticides, protein bait spraying [6], trapping in the contaminated environment [7] and sterilization of male insects [8]. Currently, this pest is controlled by using trapping or spraying methods in some gardens of Mazandaran province, Iran (Fig. 1).

Since using these methods has not been able to definitively prevent MFF from invading citrus fruits, early and non-destructive identification of infected fruits (in the larva stage) and prompt implementation of management activities are essential to effectively control it and prevent the birth of its next generation. Only by continuous monitoring of citrus fruits and accurate pest diagnosis, can the cost of control and crop losses be reduced [1,9]. The time of flies' attack to citrus and the onset of damage depends on the host fruit and the area's climate. However, not all farmers have timely access to experts [10]. To be able to supply the demand for agricultural products, agricultural problems should be addressed by advanced techniques. Thus, agricultural industries are focusing on the application of artificial intelligence methods [11,12,38]. Several traditional machine learning (ML) algorithms have been used to classify plant diseases. However, after the evolution of deep learning (DL), many state-of-the-art architectures, including AlexNet, Visual Geometry Group (VGG), DenseNet, Inception-v4 and ResNet, showed promising results for the classification of plant diseases [[13], [14], [15], [16]]. This is due to the automatic feature extraction capability of the DL algorithms. One of the architectures of interest in the DL method is the CNN, which has been widely used in digital image processing [[17], [18], [19]]. Among different methods used to tackle agricultural problems, the successful classification of plant diseases is vital in improving the quality/quantity of agricultural products and reducing an undesirable application of chemical sprayers such as fungicides/herbicides. Thus, advancing automated plant disease detection is an emerging research topic. However, it is a very complex process due to the resemblance of the plants affected by diseases [16]. Therefore, several studies have been conducted to improve the classification of plant diseases using CNN (Table 1). The aim of this study was to automatically detect citrus fruits infected with Mediterranean fruit fly larvae using machine vision system and convolutional neural network through transfer learning technique. Early detection of this disease in the early stages of its emergence (physical symptoms of the pest on the fruit) can minimize damage to crops and prevent the spread of the disease in orchards. Early detection of this pest can also be very effective in reducing costs in other agricultural activities.