Original papersCrop rows and weeds detection in maize fields applying a computer vision system based on geometry
Graphical abstract
Introduction
The use of machine vision systems (passive) onboard autonomous tractors become an excellent tool for site-specific applications (Gerhards and Oebel, 2006), such as weeds removal in maize fields with wide-row crops. Weeds densities and overlapping are to be detected for removal by applying herbicides or burners (Fontanelli et al., 2015, Martelloni et al., 2016, Raffaelli et al., 2015) as in the case of the RHEA (2017) project. For such purpose, a path following with a precise guidance is required, including the orientation at the starting point in the field when the tractor is ready to execute a piece of path.
Images captured in outdoor agricultural scenarios become difficult to process from the point of view of computer vision in order to measure and respond to inter and intra-field densities in crops and weeds.
Apart from the high variability and changing lighting conditions, crops are infested with weeds, shadows, soil or gravels, among others. An appropriate image preprocessing method must deal with these adverse environmental conditions. Grey levels of soil and the other non-green features can be attenuated with camera filtering and using different types of cameras, such as a grayscale camera and a near-infrared filter (Astrand and Baerveldt, 2005). The excess of green color index and Otsu’s method (Otsu, 1979) are classical approaches to attenuate effects caused by weeds pressure and environmental light changes. Besides, Meng et al. (2015) applied a HIS color model to reduce the effect of uneven illuminations on color images and produce a grayscale image with H components. They detect crop rows based on a line scanning method after image preprocessing, which is robust enough against weed noise.
To cope with the adverse environmental conditions, we have designed an image processing strategy comprising two main modules, where the geometry of the machine vision system, plays an important role. The first module is dedicated to crop rows detection with precise adjustment. It consists of three main processes: exposure time control, image segmentation and crop rows identification. Correct crop row detection is a key step to achieve the main goals proposed in our approach: (1) to reach a correct position and orientation at the starting point in the field; (2) to accomplish a precise guidance for path following and (3) to determine weeds densities in areas where specific-site treatments are to be applied and for overlapping control. The second module, identified as data extraction, consists of the corresponding processes to achieve the above these three goals. The full sequential process, from quality image acquisition to decision (weeds density matrix, guidance and overlapping) involves existing and new procedures, making the main contribution of this paper.
The exposure time control is a new strategy to acquire quality images under outdoor environmental conditions. It applies a PID controller to keep a good level of illumination in the image. Image segmentation uses the Otsu’s thresholding method (Otsu, 1979) after a greenness extraction (Guijarro et al., 2011) to separate pixels belonging to green plants (crops and weeds) from the remainder (soil, stones and others). Thresholding is a broadly technique used for image segmentation. Different approaches deal with this problem, for example, Valiente-González et al. (2014) use Principal Component Analysis to discriminate corn kernel and background pixels. On other hand, in Payne et al. (2013) the pixels were segmented into fruit and background using color segmentation in the YCbCr color model and a texture segmentation based on adjacent pixel variability. While Gong et al. (2013) estimate the yield of citrus for an individual tree using an Android mobile phone, the citrus color is determined by the score of Red and Blue values and the Green value is a constrain value for binarization and classification between fruit or background. Closer to this approach, in Guerrero et al. (2012a) crop rows and weeds were identified using Support Vector Machines applying the greenness identification described in Guijarro et al. (2011) and the Otsu’s thresholding method (Otsu, 1979). Others thresholding methods were applied in Tellaeche et al. (2008) who used the Ribeiro’s et al. (2005) method for segmentation and the Kapur’s et al. (1985) method for binarization. After segmentation and based on the system geometry, an expected crop row is mapped onto the image. Different strategies have been applied with such purpose. Guerrero et al. (2013) who applied system geometry constrains to estimate the straight crop rows. Montalvo et al. (2012) who used a double binarization technique for crop row and weeds classification, and using the geometric constrains of the camera system created some templates to apply the total least-square method for crop rows identification. And García-Santillán et al., 2017a, García-Santillán et al., 2017b, who detected curved and straight crop rows by accumulation of green pixels. The vision system is designed to be installed onboard a mobile agricultural vehicle, i.e. submitted to turns, vibrations and undesired movements. The images are captured under image perspective projection, being affected by the above undesired effects, and the expected crop rows could differ from the real ones. Thus, a correction process, based on the Theil-Sen estimator (Theil, 1950, Sen, 1968, Guerrero et al., 2013) has been applied to correct the expected line in order to approximate the expected crop row to the real one. The performance of this estimator has been statistically verified being applied for crop row detection in terms of accuracy (Guerrero et al., 2013).
The paper is organized as follows. Section 2 describes the software design for the machine vision, based on image processing techniques. The system architecture is proposed and its modules explained. Section 3 provides the significant results obtained with comparative performances, including the full strategy and some processes belonging to crop rows detection modules. Finally, Section 4 provides the relevant conclusions.
Section snippets
System architecture
The software architecture is inspired on the human expert knowledge based on the specific application. Astrand and Baerveldt, 2005, Slaughter et al., 2008 provide a list of requirements for guidance systems that can be also considered for crop row detection, which is a similar problem.
Based on this knowledge and requirements and considering advantages and shortcomings of the different crop row detection methods existing in the literature as expressed above, the automatic software processing is
Results
Two sets of images, belonging to maize crops, have been used for testing: (1) SET-1: with 1847 images captured with a SVS-VISTEK SVS4050CFLGEA (2017) color camera with a resolution of 2336 x 1752 pixels and saved in RGB (Red, Green and Blue) color space in the BMP format. This was the camera used on the RHEA project. (2) SET-2: with 700 images captured with the BASLER (2017) scA1400-17FC color camera with resolutions of 1392 × 1038 pixels saved in RGB (Red, Green and Blue) color space in the TIFF
Conclusions
The main contribution of this research lies in the design of an image processing architecture for PA, able to detect crop rows and weeds in maize fields, to correct deviations during tractor guidance and to control the overlapping of the areas to be treated. The system is based on two main modules. The first is in charge of crop row detection, obtaining high quality images, its segmentation to obtain binary images where plants appear as white pixels and the remainder in black and to estimate
Acknowledgements
The research has been inspired and partially funded as part of the RHEA project funded by the European Union's Seventh Framework Programme [FP7/2007e2013] under Grant Agreement no. 245986 in the Theme FP7 Nanosciences, Nanotechnologies, Materials and new Production Technologies-2009-3.4-1 (Automation and robotics for sustainable crop and forestry management).
References (39)
- et al.
A vision based row-following system for agricultural field machinery
Mechatronics
(2005) - et al.
Automatic detection of curved and straight crop rows from images in maize fields
Biosyst. Eng.
(2017) - et al.
Citrus yield estimation based on images processed by an android mobile phone
Biosys. Eng.
(2013) - et al.
Automatic expert system based on images for accuracy crop row detection in maize fields
Expert Syst. Appl.
(2013) - et al.
Support vector machines for crop/weeds identification in maize fields
Expert Syst. Appl.
(2012) - et al.
Automatic segmentation of relevant textures in agricultural images
Comput. Electron. Agric.
(2011) - et al.
A new method for gray-level picture thresholding using the entropy of the histogram
Comput. Vision Graph. Image Process
(1985) - et al.
Overview of total least squares methods
Signal Process.
(2007) - et al.
Development of agricultural implement system based on machine vision and fuzzy control
Comput. Electron. Agric.
(2015) - et al.
Verification of color vegetation indices for automated crop imaging applications
Comput. Electron. Agric.
(2008)
Automatic detection of crop rows in maize fields with high weeds pressure
Expert Syst. Appl.
Estimation of mango crop yield using image analysis-segmentation method
Comput. Electron. Agric.
Autonomous robotic weed control systems: A review
Comput. Electron. Agric.
A vision-based method for weeds identification through the bayesian decision theory
Pattern Recogn.
Automatic corn (Zea mays) kernel inspection system using novelty detection based on principal component analysis
Biosys. Eng.
Fleets of robots for precision agriculture: a simulation environment
Ind. Robot: Int. J.
Innovative strategies and machines for physical weed control in organic and integrated vegetable crops
Chem. Eng. Trans.
Cited by (49)
Site-specific herbicide spraying can control weeds as well as full spraying in the long-term. A simulation study
2023, Computers and Electronics in AgricultureEco-friendly weeding through precise detection of growing points via efficient multi-branch convolutional neural networks
2023, Computers and Electronics in AgricultureVision-based navigation and guidance for agricultural autonomous vehicles and robots: A review
2023, Computers and Electronics in AgricultureWeed detection in canola fields using maximum likelihood classification and deep convolutional neural network
2020, Information Processing in AgricultureCitation Excerpt :They classify any interline vegetation as weeds [12]. Another crop line detection method involves using geometric constraints for mapping regions of interest [18]. Intra-line approaches attempt to classify weeds and crops irrespective of their position in the field.