Original papers
Crop rows and weeds detection in maize fields applying a computer vision system based on geometry

https://doi.org/10.1016/j.compag.2017.09.028Get rights and content

Highlights

  • We design a method to guide a tractor correctly at the starting point in the field.

  • We determine weeds density applying the system geometry.

  • We control the overlapping of the treated areas based on a GPS.

  • We design an automatic method to guide the tractor in real-time during the treatment.

Abstract

This paper proposes a new machine vision system, focused on crop row detection with three main goals: (1) to achieve a correct orientation at the starting point in the field for each path; (2) to achieve a precise guidance during path following and (3) to determine the weed density and overlapping. The vision system is designed to be installed onboard a mobile agricultural vehicle, i.e. submitted to turns, vibrations and undesired movements. The images are captured under image perspective projection, being affected by the above undesired effects. To achieve the above goals, we have designed an image processing strategy comprising two main modules: (a) crop rows detection with precise adjustment and (b) data extraction for determining weeds densities for site specific-treatments, guidance and the overlapping of the area where the treatment is to be applied. Both modules form a full process, and each module contains different processes, some of them coming from existing approaches (image segmentation and crop row estimation with adjustment) and other specifically designed for the proposed approach (exposure time control, position and orientation of the tractor and weeds density matrix computation with overlapping control).

The proposed strategy has been conveniently tested and also the different processes defining each module, verifying their performances in terms of accuracies and processing times. Comparative analyses have also been carried out against some existing processes, also with acceptable performances.

Introduction

The use of machine vision systems (passive) onboard autonomous tractors become an excellent tool for site-specific applications (Gerhards and Oebel, 2006), such as weeds removal in maize fields with wide-row crops. Weeds densities and overlapping are to be detected for removal by applying herbicides or burners (Fontanelli et al., 2015, Martelloni et al., 2016, Raffaelli et al., 2015) as in the case of the RHEA (2017) project. For such purpose, a path following with a precise guidance is required, including the orientation at the starting point in the field when the tractor is ready to execute a piece of path.

Images captured in outdoor agricultural scenarios become difficult to process from the point of view of computer vision in order to measure and respond to inter and intra-field densities in crops and weeds.

Apart from the high variability and changing lighting conditions, crops are infested with weeds, shadows, soil or gravels, among others. An appropriate image preprocessing method must deal with these adverse environmental conditions. Grey levels of soil and the other non-green features can be attenuated with camera filtering and using different types of cameras, such as a grayscale camera and a near-infrared filter (Astrand and Baerveldt, 2005). The excess of green color index and Otsu’s method (Otsu, 1979) are classical approaches to attenuate effects caused by weeds pressure and environmental light changes. Besides, Meng et al. (2015) applied a HIS color model to reduce the effect of uneven illuminations on color images and produce a grayscale image with H components. They detect crop rows based on a line scanning method after image preprocessing, which is robust enough against weed noise.

To cope with the adverse environmental conditions, we have designed an image processing strategy comprising two main modules, where the geometry of the machine vision system, plays an important role. The first module is dedicated to crop rows detection with precise adjustment. It consists of three main processes: exposure time control, image segmentation and crop rows identification. Correct crop row detection is a key step to achieve the main goals proposed in our approach: (1) to reach a correct position and orientation at the starting point in the field; (2) to accomplish a precise guidance for path following and (3) to determine weeds densities in areas where specific-site treatments are to be applied and for overlapping control. The second module, identified as data extraction, consists of the corresponding processes to achieve the above these three goals. The full sequential process, from quality image acquisition to decision (weeds density matrix, guidance and overlapping) involves existing and new procedures, making the main contribution of this paper.

The exposure time control is a new strategy to acquire quality images under outdoor environmental conditions. It applies a PID controller to keep a good level of illumination in the image. Image segmentation uses the Otsu’s thresholding method (Otsu, 1979) after a greenness extraction (Guijarro et al., 2011) to separate pixels belonging to green plants (crops and weeds) from the remainder (soil, stones and others). Thresholding is a broadly technique used for image segmentation. Different approaches deal with this problem, for example, Valiente-González et al. (2014) use Principal Component Analysis to discriminate corn kernel and background pixels. On other hand, in Payne et al. (2013) the pixels were segmented into fruit and background using color segmentation in the YCbCr color model and a texture segmentation based on adjacent pixel variability. While Gong et al. (2013) estimate the yield of citrus for an individual tree using an Android mobile phone, the citrus color is determined by the score of Red and Blue values and the Green value is a constrain value for binarization and classification between fruit or background. Closer to this approach, in Guerrero et al. (2012a) crop rows and weeds were identified using Support Vector Machines applying the greenness identification described in Guijarro et al. (2011) and the Otsu’s thresholding method (Otsu, 1979). Others thresholding methods were applied in Tellaeche et al. (2008) who used the Ribeiro’s et al. (2005) method for segmentation and the Kapur’s et al. (1985) method for binarization. After segmentation and based on the system geometry, an expected crop row is mapped onto the image. Different strategies have been applied with such purpose. Guerrero et al. (2013) who applied system geometry constrains to estimate the straight crop rows. Montalvo et al. (2012) who used a double binarization technique for crop row and weeds classification, and using the geometric constrains of the camera system created some templates to apply the total least-square method for crop rows identification. And García-Santillán et al., 2017a, García-Santillán et al., 2017b, who detected curved and straight crop rows by accumulation of green pixels. The vision system is designed to be installed onboard a mobile agricultural vehicle, i.e. submitted to turns, vibrations and undesired movements. The images are captured under image perspective projection, being affected by the above undesired effects, and the expected crop rows could differ from the real ones. Thus, a correction process, based on the Theil-Sen estimator (Theil, 1950, Sen, 1968, Guerrero et al., 2013) has been applied to correct the expected line in order to approximate the expected crop row to the real one. The performance of this estimator has been statistically verified being applied for crop row detection in terms of accuracy (Guerrero et al., 2013).

The paper is organized as follows. Section 2 describes the software design for the machine vision, based on image processing techniques. The system architecture is proposed and its modules explained. Section 3 provides the significant results obtained with comparative performances, including the full strategy and some processes belonging to crop rows detection modules. Finally, Section 4 provides the relevant conclusions.

Section snippets

System architecture

The software architecture is inspired on the human expert knowledge based on the specific application. Astrand and Baerveldt, 2005, Slaughter et al., 2008 provide a list of requirements for guidance systems that can be also considered for crop row detection, which is a similar problem.

Based on this knowledge and requirements and considering advantages and shortcomings of the different crop row detection methods existing in the literature as expressed above, the automatic software processing is

Results

Two sets of images, belonging to maize crops, have been used for testing: (1) SET-1: with 1847 images captured with a SVS-VISTEK SVS4050CFLGEA (2017) color camera with a resolution of 2336 x 1752 pixels and saved in RGB (Red, Green and Blue) color space in the BMP format. This was the camera used on the RHEA project. (2) SET-2: with 700 images captured with the BASLER (2017) scA1400-17FC color camera with resolutions of 1392 × 1038 pixels saved in RGB (Red, Green and Blue) color space in the TIFF

Conclusions

The main contribution of this research lies in the design of an image processing architecture for PA, able to detect crop rows and weeds in maize fields, to correct deviations during tractor guidance and to control the overlapping of the areas to be treated. The system is based on two main modules. The first is in charge of crop row detection, obtaining high quality images, its segmentation to obtain binary images where plants appear as white pixels and the remainder in black and to estimate

Acknowledgements

The research has been inspired and partially funded as part of the RHEA project funded by the European Union's Seventh Framework Programme [FP7/2007e2013] under Grant Agreement no. 245986 in the Theme FP7 Nanosciences, Nanotechnologies, Materials and new Production Technologies-2009-3.4-1 (Automation and robotics for sustainable crop and forestry management).

References (39)

  • M. Montalvo et al.

    Automatic detection of crop rows in maize fields with high weeds pressure

    Expert Syst. Appl.

    (2012)
  • A.B. Payne et al.

    Estimation of mango crop yield using image analysis-segmentation method

    Comput. Electron. Agric.

    (2013)
  • D.C. Slaughter et al.

    Autonomous robotic weed control systems: A review

    Comput. Electron. Agric.

    (2008)
  • A. Tellaeche et al.

    A vision-based method for weeds identification through the bayesian decision theory

    Pattern Recogn.

    (2008)
  • J.M. Valiente-González et al.

    Automatic corn (Zea mays) kernel inspection system using novelty detection based on principal component analysis

    Biosys. Eng.

    (2014)
  • BASLER, 2017. BASLER the power of sight. Available online: <http://www.baslerweb.com/en> (acceded in March...
  • cRIO-9082, 2017. High Performance Integrated System. Available online:...
  • L. Emmi et al.

    Fleets of robots for precision agriculture: a simulation environment

    Ind. Robot: Int. J.

    (2013)
  • M. Fontanelli et al.

    Innovative strategies and machines for physical weed control in organic and integrated vegetable crops

    Chem. Eng. Trans.

    (2015)
  • Cited by (49)

    • Weed detection in canola fields using maximum likelihood classification and deep convolutional neural network

      2020, Information Processing in Agriculture
      Citation Excerpt :

      They classify any interline vegetation as weeds [12]. Another crop line detection method involves using geometric constraints for mapping regions of interest [18]. Intra-line approaches attempt to classify weeds and crops irrespective of their position in the field.

    View all citing articles on Scopus
    View full text