Original papersEnd-effector with a bite mode for harvesting citrus fruit in random stalk orientation environment
Graphical abstract
Prototype of end-effector for citrus harvesting robot.
Introduction
Since the idea of robotic harvesting was put forward by Schertz and Brown in 1968, the harvesting robots have been extensively studied under the motivation of social, economic, environmental and food quality factors (Zhao et al., 2016a, Zhao et al., 2016b, Bac et al., 2016). However, they are still studied mainly in the laboratory due to the lack of robust fruit recognition and precision harvesting capability (Zhao et al., 2016a, Zhao et al., 2016b, Bechar and Vigneault, 2016). Since the computing power of computer is improved, the recognition systems of harvesting robots have been broadly studied in recent years, and some of them have been successfully developed to harvest the fruits and vegetables, such as apple (Liu et al., 2016, Si et al., 2015, Nguyen et al., 2016), cherry (Kaczmarek, 2017, Amatya et al., 2016), strawberry (Kaczmarek et al., 2017), citrus (Mehta and Burks, 2014, Lu and Sang, 2015, Lu et al., 2018), grape clusters (Luo et al., 2016, Luo et al., 2018), litchi (Wang et al., 2016a, Wang et al., 2016b, Xiong et al., 2018), tomato (Zhao et al., 2016a, Zhao et al., 2016b), cucumber (Bao et al., 2016), asparagus (Sakai et al., 2013), broccoli (Pieter et al., 2016) and pepper (Bac et al., 2014a, Bac et al., 2014b, Lehnert et al., 2016, Eizentals and Oka, 2016). Furthermore, various end-effectors have been developed to enhance the harvesting capability of harvesting robots (Zhong et al., 2015, Zhao et al., 2011, Wang et al., 2016a, Wang et al., 2016b, Kondo et al., 2010, Jia et al., 2009, Davidson et al., 2016 Li et al., 2016, Liu et al., 2013).
The harvesting processes involve the fitting of shapes of fruits and vegetables and the detection of cutting points by the recognition systems (Eizentals and Oka, 2016, Lehnert et al., 2016, Luo et al., 2018, Xiong et al., 2018), which can be affected by environmental conditions such as obscuration of leaves, swing of fruits and vegetables caused by wind, and machine collision (Xiong et al., 2018), etc. As a consequence, the failure of detection results in the reduction of harvesting rate. In other words, it is difficult for the recognition system to accurately detect the cutting points of fruits and vegetables grown in the natural environment. Thus, we aim to improve the harvesting capacity of harvesting robot through other methods such as improving the end-effector and adjusting the harvesting postures.
To improve the harvesting capability of harvesting robots, the harvesting rate for fruits and vegetables must be enhanced. The end-effector, as a critical component of a harvesting robot, plays a determinative role in the process of harvesting the fruits and vegetables. In other words, the harvesting efficiency of harvesting robots is closely related to the properties of end-effector (Bac et al., 2014a, Bac et al., 2014b). Due to the complex environment, random shapes and stalk orientations of fruits and vegetables, the end-effector must accurately and flexibly harvest them to ensure the result of harvesting. Although there are some reports on the end-effectors of harvesting robots, they are mainly used to harvest fruits and vegetables grown in laboratories or greenhouses (Zhong et al., 2015, Wang et al., 2016a, Wang et al., 2016b, Kondo et al., 2010, Jia et al., 2009, Li et al., 2016, Liu et al., 2013). It is known that the growth of fruits and vegetables in laboratories or greenhouses can be controlled, thus the structures of end-effectors are relatively simple. For those fruits grown in natural environment such as citrus fruits, the random shapes and orientations make them be harvested difficultly. To successfully harvest them, the properties of end-effector must be improved, i.e. the end-effectors require further development.
From the results reported on the end-effectors, they usually cut the fruit stalks in the modes of shearing by scissors, grasp-kinking by three fingers of robots and grasp-cutting by the harvesting mechanisms of robots (Kondo et al., 2010, Jia et al., 2009, Davidson et al., 2016, Li et al., 2016, Liu et al., 2013). In these references, the authors told that the end-effectors should be improved to well harvest the fruits. Through analysis of the three modes, we think that they are not appropriate for harvesting the citrus fruits grown in natural environment because the shearing mode is adapted to harvest the oriented fruits. Both the grasp-kinking and grasp-cutting modes may produce some damages for the fruits in the process of grasping them. Therefore, it is necessary to design an end-effector which can harvest the citrus fruits with random stalk orientations and reduce the damage of citrus fruits.
When the industrial robots are applied, it is found that the efficiency of robots is closely related to the postures of robots, thus the optimization of operating postures is extensively studied (Lin et al., 2017, Zargarbashi et al., 2012, Guo et al., 2015, Caro et al., 2013). How about the agricultural robots? There are a few of reports on the studies of postures of end-effectors. Bac et al. (2016) studied the posture of end-effector and determined the posture by the orientation information of sweet-pepper stems at the pre-picking location. However, the posture of end-effector can not be determined until the orientations of sweet-peppers are recognized. This indicates that the studies of Bac et al. can be applied in the environment without obstacles. In fact, some fruits are usually covered by the leaves in the natural environment, which results in the difficulty of recognizing the orientations of fruits. Thus, the postures of end-effectors require further optimization.
Based on the above mentioned statements, the end-effector and its postures require improvement and optimization in order to realize the successful harvesting of citrus fruits with random stalk orientations. For the improvement of end-effector and optimization of harvesting postures, we built a harvesting coordinate system, defined some angles and designed an end-effector with a biting function by simulating the head mechanism of snake. According to the designed end-effector, a method of optimizing harvesting postures was proposed and an evaluation function of harvesting postures was obtained. The designed end-effector and optimized angle were applied to the practical harvesting of citrus fruits, which indicated that the design of end-effector and the method of optimizing harvesting postures were acceptable.
The paper is structured as follows. In Section 2, we introduced the harvesting coordinate system and some angles to describe the stalk orientation of a citrus, and described the design of end-effector with a biting function and the optimization of harvesting postures. Section 3 presents the experimental evidences for the designed end-effector and the optimization of harvesting postures, which were carried out in laboratory and natural environment. Finally, the paper concludes with a brief summary.
Section snippets
Determination of stalk orientations of citrus fruits
It is known that the growth of citrus fruits in natural environment is random, which results in the random stalk orientations of citrus fruits. The robot used for harvesting the citrus fruits is shown in Fig. 1. The monocular camera is used to obtain the information of the orchard road, and then the information is transmitted to IPC to determine the harvesting route of harvesting robot. The binocular camera is used to acquire the images of objective citrus for determining the center coordinates
Results of cutting the citrus stalks with different diameters and deflection angles
Because the maximum work pressure of MHY2-16D finger cylinder is 0.6 MPa, the cutting experiments are carried out using the pressure of 0.6 MPa. During the experiments, the stalks are divided into 6 groups depending on their diameters shown in Fig. 5(c), where the diameter interval of each group in a range of 2–4 mm is 0.5 mm. The deflection angle range in [−50°, 50°] is divided into 5 groups shown in Table 2. There are 30 group stalks used for the cutting experiment and each group includes 20
Conclusions
Depending on the bionic principle, an end-effector with the bite mode for citrus harvesting robot is designed by simulating the head mechanism of snake. According to the end-effector, a method of optimizing harvesting postures is suggested and a function to evaluate the harvesting postures is obtained. Combining the structural parameters of end-effector and the growth orientations of citrus fruits, the evaluating function is applied and the optimal posture of 46°is obtained.
According to the
Acknowledgements
This work was supported by the basic science and frontier technology research (general) project (cstc2018jcyjAX0071) and the basic science and frontier technology research project (cstc2016jcyja0444) of Chongqing, China.
References (39)
- et al.
Detection of cherry tree branches with full foliage in planar architecture for automated sweet-cherry harvesting
Biosystems Eng.
(2016) - et al.
Stem localization of sweet-pepper plants using the support wire as a visual cue
Comput. Electron. Agric.
(2014) - et al.
Analysis of a motion planning problem for sweet pepper harvesting in a dense obstacle environment
Biosystems Eng.
(2016) - et al.
Multi-template matching algorithm for cucumber recognition in natural environment
Comput. Electron. Agric.
(2016) - et al.
Agricultural robots for field operations: concepts and components
Biosystems Eng.
(2016) - et al.
3D pose estimation of green pepper fruit for automated harvesting
Comput. Electron. Agric.
(2016) - et al.
Stiffness-oriented posture optimization in robotic machining applications
Robot. Computer-Integrated Manuf.
(2015) Stereo vision with Equal Baseline Multiple Camera Set (EBMCS) for obtaining depth maps of plants
Comput. Electron. Agric.
(2017)- et al.
Development of an end-effector for a tomato cluster harvesting robot
Eng. Agric. Environ. Food
(2010) - et al.
Characterizing apple picking patterns for robotic harvesting
Comput. Electron. Agric.
(2016)
Posture optimization methodology of 6R industrial robots for machining using performance evaluation indexes
Robot. Comput.-Integrated Manuf.
A method of segmenting apples at night based on color and position information
Comput. Electron. Agric.
Immature citrus fruit detection based on local binary pattern feature and hierarchical contour analysis
Biosystems Eng.
Detecting citrus fruits and occlusion recovery under natural illumination conditions
Comput. Electron. Agric.
Vision-based extraction of spatial information in grape clusters for harvesting robots
Biosystems Eng.
A vision methodology for harvesting robot to detect cutting points on peduncles of double overlapping grape clusters in a vineyard
Comput. Industry
Vision-based control of robotic manipulator for citrus harvesting
Comput. Electron. Agric.
Detection of red and bicoloured apples on tree with an RGB-D camera
Biosystems Eng.
Accurate position detecting during asparagus spear harvesting using a laser sensor
Eng. Agric. Environ. Food
Cited by (53)
A comb-type end-effector for inflorescence thinning of table grapes
2024, Computers and Electronics in AgricultureExperiment and analysis on walnut (Juglans regia L.) shedding force based on low-frequency vibration response
2023, Industrial Crops and ProductsDesign of a novel end-effector for robotic bud thinning of Agaricus bisporus mushrooms
2023, Computers and Electronics in AgricultureA review of core agricultural robot technologies for crop productions
2023, Computers and Electronics in AgricultureA review of the large-scale application of autonomous mobility of agricultural platform
2023, Computers and Electronics in AgricultureA review on multirobot systems in agriculture
2022, Computers and Electronics in Agriculture