Towards Food Handling Robots for Automated Meal Preparation in Healthcare Facilities

Abstract

Meal preparation in healthcare facilities is an area of work severely affected by the shortage of qualified personnel. Recent advances in automation technology have enabled the use of picking robots to fill the gaps created by changes in the labor market. Building on these advances, we present a robotic system designed to handle packaged food for automated meal preparation in healthcare facilities. To address the challenge of grasping the large variety of packaged foods, we propose a novel technique for model-free grasping pose detection based on geometric features that is optimized for the given scenario. We provide a comprehensive system overview and conduct evaluations of the grasping success on a real robot. The high grasping success rate of $94\%$ with a processing time of $\sim \!\!280$ ms indicates the suitability of the proposed approach for automating meal preparation tasks in healthcare facilities.

Appendix

We provide an ablation and hyperparameter study for the components of the grasping pose detection method. The experimental conditions and setup are identical to the evaluations reported in Sect. 5. A reduced number of 100 grasping cycles are performed per experiment, and the grasping success rate as well as the mean $\mu$ and standard deviation $\sigma$ of the processing time for each cycle, including image retrieval and grasp computation, are evaluated. All grasp pose detection calculations are performed on an Nvidia RTX3090 GPU. During each cycle, the grasp with the highest score is executed first. If this is unsuccessful, up to two more grasp attempts are made without recomputing the grasps to measure both the top-1 and top-3 accuracy.

For the component ablation study, the algorithm is evaluated when grasp rejection via the proposed ray-casting approach is not used. In addition, the top-3 success rate is measured when the grasp set is not pruned and all grasping poses whose distance to other candidates is smaller than the suction cup diameter are retained. The results are shown in Table 2.

Table 2. Grasping evaluation results for the components ablation study.

We observe that the grasping success rate without the ray-casting rejection approach drops to a top-1 performance of $77\%$ with a reduced processing time of $\sim \!\!170$ ms. The failed grasps here are attributed to noise in the camera image leading to invalid grasps, which are prevented by checking the local area against the suction cup in the ray-casting process. For the experiment without pruning of the grasp set, we see that the top-3 performance slightly declines to $97\%$ with a negligible processing time difference in the range of microseconds. Thus, the results of the ablation study show the positive impact of the developed algorithm components.

For the hyperparameter study of the grasping detection process, the impact of changes in individual parameters on the success rate and processing time is investigated, with all other parameters remaining unchanged. The optimal hyperparameters used in the results in Sect. 5 are: neighborhood size $A=5\times 5$, threshold $c=0.9$, and iterations $T=5$. The results for the varied parameters A, c and T are shown in Table 3.

Table 3. Grasping evaluation results for the hyperparameter study.

We observe that when varying the neighborhood size A, the grasping results show negligible differences in both success rates and speed, indicating that any of the neighborhood sizes can be chosen for successful grasping. For the threshold parameter c, on the other hand, we find a strong difference in success rates: for smaller thresholds $c=0.7$, the top-1 success rate drops to $83\%$. This is because with smaller thresholds, the object edges are smoothed by the heuristic and the proposed grasps are predicted closer to the edges, leading to more failed grasps. For larger thresholds $c=0.95$, small irregularities, e.g., on the lid of objects, lead to strongly varying grasp quality values with many local maxima that are not centered on the objects and thus can lead to unstable grasps. Finally, for the number of iterations T the heuristic is applied to the scene, we find that the grasping success declines for both a higher or lower number of iterations. We assume that this hyperparameter, which depends on the object sizes in pixels, must be tuned individually for the given camera intrinsic parameters and distance of the camera from the objects.