Original papers
End-effector with a bite mode for harvesting citrus fruit in random stalk orientation environment

https://doi.org/10.1016/j.compag.2019.01.015Get rights and content

Highlights

  • Introduction of new descriptions for the stalk orientations of citruses.

  • Design of end-effector with bite mode by simulating mechanism of snake head.

  • A function evaluating harvesting postures.

  • Harvesting of citruses with random stalk orientations.

Abstract

It is difficult for harvesting robot to harvest the citrus fruits with random stalk orientations. To harvest citrus fruits with random stalk orientations by harvesting robot, some new descriptions for the stalk orientations of citrus fruits are introduced and an end-effector with the bite mode for citrus harvesting robot is designed by simulating the head mechanism of snake. According to the end-effector, an optimization method of harvesting postures is suggested and a function to evaluate the harvesting postures is obtained. Combined with the structural parameters of end-effector and the growth orientations of factual citrus fruits, the evaluation function is applied in the harvesting of citrus fruits grown in the natural environment and the optimal harvesting posture of 46° is obtained. Based on the model of end-effector, the end-effector prototype with the bite mode is assembled and then applied in a harvesting robot. The results of cutting citrus stalks and harvesting citrus experiments in laboratory indicate that the average success rate of cutting is up to about 98% in the deflection angle range of [−50°,50°] and the harvesting rate in the optimal posture is about 89%. In addition, the harvesting results in natural environments show that the harvesting rate in the optimal posture is up to 74%. The harvesting experiments in both laboratory and natural environments demonstrate that the harvesting rate in the optimal posture is higher than other harvesting posture. The results show the designed end-effector and the method of optimizing harvesting postures are acceptable and the results can contribute to the development of next-generation citrus harvesting robots.

Graphical abstract

Prototype of end-effector for citrus harvesting robot.

  1. Download : Download high-res image (48KB)
  2. Download : Download full-size image

Introduction

Since the idea of robotic harvesting was put forward by Schertz and Brown in 1968, the harvesting robots have been extensively studied under the motivation of social, economic, environmental and food quality factors (Zhao et al., 2016a, Zhao et al., 2016b, Bac et al., 2016). However, they are still studied mainly in the laboratory due to the lack of robust fruit recognition and precision harvesting capability (Zhao et al., 2016a, Zhao et al., 2016b, Bechar and Vigneault, 2016). Since the computing power of computer is improved, the recognition systems of harvesting robots have been broadly studied in recent years, and some of them have been successfully developed to harvest the fruits and vegetables, such as apple (Liu et al., 2016, Si et al., 2015, Nguyen et al., 2016), cherry (Kaczmarek, 2017, Amatya et al., 2016), strawberry (Kaczmarek et al., 2017), citrus (Mehta and Burks, 2014, Lu and Sang, 2015, Lu et al., 2018), grape clusters (Luo et al., 2016, Luo et al., 2018), litchi (Wang et al., 2016a, Wang et al., 2016b, Xiong et al., 2018), tomato (Zhao et al., 2016a, Zhao et al., 2016b), cucumber (Bao et al., 2016), asparagus (Sakai et al., 2013), broccoli (Pieter et al., 2016) and pepper (Bac et al., 2014a, Bac et al., 2014b, Lehnert et al., 2016, Eizentals and Oka, 2016). Furthermore, various end-effectors have been developed to enhance the harvesting capability of harvesting robots (Zhong et al., 2015, Zhao et al., 2011, Wang et al., 2016a, Wang et al., 2016b, Kondo et al., 2010, Jia et al., 2009, Davidson et al., 2016 Li et al., 2016, Liu et al., 2013).

The harvesting processes involve the fitting of shapes of fruits and vegetables and the detection of cutting points by the recognition systems (Eizentals and Oka, 2016, Lehnert et al., 2016, Luo et al., 2018, Xiong et al., 2018), which can be affected by environmental conditions such as obscuration of leaves, swing of fruits and vegetables caused by wind, and machine collision (Xiong et al., 2018), etc. As a consequence, the failure of detection results in the reduction of harvesting rate. In other words, it is difficult for the recognition system to accurately detect the cutting points of fruits and vegetables grown in the natural environment. Thus, we aim to improve the harvesting capacity of harvesting robot through other methods such as improving the end-effector and adjusting the harvesting postures.

To improve the harvesting capability of harvesting robots, the harvesting rate for fruits and vegetables must be enhanced. The end-effector, as a critical component of a harvesting robot, plays a determinative role in the process of harvesting the fruits and vegetables. In other words, the harvesting efficiency of harvesting robots is closely related to the properties of end-effector (Bac et al., 2014a, Bac et al., 2014b). Due to the complex environment, random shapes and stalk orientations of fruits and vegetables, the end-effector must accurately and flexibly harvest them to ensure the result of harvesting. Although there are some reports on the end-effectors of harvesting robots, they are mainly used to harvest fruits and vegetables grown in laboratories or greenhouses (Zhong et al., 2015, Wang et al., 2016a, Wang et al., 2016b, Kondo et al., 2010, Jia et al., 2009, Li et al., 2016, Liu et al., 2013). It is known that the growth of fruits and vegetables in laboratories or greenhouses can be controlled, thus the structures of end-effectors are relatively simple. For those fruits grown in natural environment such as citrus fruits, the random shapes and orientations make them be harvested difficultly. To successfully harvest them, the properties of end-effector must be improved, i.e. the end-effectors require further development.

From the results reported on the end-effectors, they usually cut the fruit stalks in the modes of shearing by scissors, grasp-kinking by three fingers of robots and grasp-cutting by the harvesting mechanisms of robots (Kondo et al., 2010, Jia et al., 2009, Davidson et al., 2016, Li et al., 2016, Liu et al., 2013). In these references, the authors told that the end-effectors should be improved to well harvest the fruits. Through analysis of the three modes, we think that they are not appropriate for harvesting the citrus fruits grown in natural environment because the shearing mode is adapted to harvest the oriented fruits. Both the grasp-kinking and grasp-cutting modes may produce some damages for the fruits in the process of grasping them. Therefore, it is necessary to design an end-effector which can harvest the citrus fruits with random stalk orientations and reduce the damage of citrus fruits.

When the industrial robots are applied, it is found that the efficiency of robots is closely related to the postures of robots, thus the optimization of operating postures is extensively studied (Lin et al., 2017, Zargarbashi et al., 2012, Guo et al., 2015, Caro et al., 2013). How about the agricultural robots? There are a few of reports on the studies of postures of end-effectors. Bac et al. (2016) studied the posture of end-effector and determined the posture by the orientation information of sweet-pepper stems at the pre-picking location. However, the posture of end-effector can not be determined until the orientations of sweet-peppers are recognized. This indicates that the studies of Bac et al. can be applied in the environment without obstacles. In fact, some fruits are usually covered by the leaves in the natural environment, which results in the difficulty of recognizing the orientations of fruits. Thus, the postures of end-effectors require further optimization.

Based on the above mentioned statements, the end-effector and its postures require improvement and optimization in order to realize the successful harvesting of citrus fruits with random stalk orientations. For the improvement of end-effector and optimization of harvesting postures, we built a harvesting coordinate system, defined some angles and designed an end-effector with a biting function by simulating the head mechanism of snake. According to the designed end-effector, a method of optimizing harvesting postures was proposed and an evaluation function of harvesting postures was obtained. The designed end-effector and optimized angle were applied to the practical harvesting of citrus fruits, which indicated that the design of end-effector and the method of optimizing harvesting postures were acceptable.

The paper is structured as follows. In Section 2, we introduced the harvesting coordinate system and some angles to describe the stalk orientation of a citrus, and described the design of end-effector with a biting function and the optimization of harvesting postures. Section 3 presents the experimental evidences for the designed end-effector and the optimization of harvesting postures, which were carried out in laboratory and natural environment. Finally, the paper concludes with a brief summary.

Section snippets

Determination of stalk orientations of citrus fruits

It is known that the growth of citrus fruits in natural environment is random, which results in the random stalk orientations of citrus fruits. The robot used for harvesting the citrus fruits is shown in Fig. 1. The monocular camera is used to obtain the information of the orchard road, and then the information is transmitted to IPC to determine the harvesting route of harvesting robot. The binocular camera is used to acquire the images of objective citrus for determining the center coordinates

Results of cutting the citrus stalks with different diameters and deflection angles

Because the maximum work pressure of MHY2-16D finger cylinder is 0.6 MPa, the cutting experiments are carried out using the pressure of 0.6 MPa. During the experiments, the stalks are divided into 6 groups depending on their diameters shown in Fig. 5(c), where the diameter interval of each group in a range of 2–4 mm is 0.5 mm. The deflection angle range in [−50°, 50°] is divided into 5 groups shown in Table 2. There are 30 group stalks used for the cutting experiment and each group includes 20

Conclusions

Depending on the bionic principle, an end-effector with the bite mode for citrus harvesting robot is designed by simulating the head mechanism of snake. According to the end-effector, a method of optimizing harvesting postures is suggested and a function to evaluate the harvesting postures is obtained. Combining the structural parameters of end-effector and the growth orientations of citrus fruits, the evaluating function is applied and the optimal posture of 46°is obtained.

According to the

Acknowledgements

This work was supported by the basic science and frontier technology research (general) project (cstc2018jcyjAX0071) and the basic science and frontier technology research project (cstc2016jcyja0444) of Chongqing, China.

References (39)

Cited by (53)

  • A comb-type end-effector for inflorescence thinning of table grapes

    2024, Computers and Electronics in Agriculture
  • A review of core agricultural robot technologies for crop productions

    2023, Computers and Electronics in Agriculture
  • A review on multirobot systems in agriculture

    2022, Computers and Electronics in Agriculture
View all citing articles on Scopus
View full text