Abstract
Bin picking based on deep learning techniques is a promising approach that can solve several analytical methods problems. These systems can provide accurate solutions to bin picking in cluttered environments, where the scenario is always changing. This article proposes a robust and accurate system for segmenting bin picking objects, employing an easy configuration procedure to adjust the framework according to a specific object. The framework is implemented in Robot Operating System (ROS) and is divided into a detection and segmentation system. The detection system employs Mask R-CNN instance neural network to identify several objects from two dimensions (2D) grayscale images. The segmentation system relies on the point cloud library (PCL), manipulating 3D point cloud data according to the detection results to select particular points of the original point cloud, generating a partial point cloud result. Furthermore, to complete the bin picking system a pose estimation approach based on matching algorithms is employed, such as Iterative Closest Point (ICP). The system was evaluated for two types of objects, knee tube, and triangular wall support, in cluttered environments. It displayed an average precision of 79% for both models, an average recall of 92%, and an average IOU of 89%. As exhibited throughout the article, this system demonstrates high accuracy in cluttered environments with several occlusions for different types of objects.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Doumanoglou, A., Kouskouridas, R., Malassiotis, S., Kim, T.-K.: Recovering 6d object pose and predicting next-best-view in the crowd. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Jun 2016. https://doi.org/10.1109/cvpr.2016.390
Pochyly, A., Kubela, T., Singule, V., Cihak, P.: Robotic bin-picking system based on a revolving vision system. In: 2017 19th International Conference on Electrical Drives and Power Electronics (EDPE). IEEE, Oct 2017. https://doi.org/10.1109/edpe.2017.8123228
Choi, C., Taguchi, Y., Tuzel, O., Liu, M.-Y., Ramalingam, S.: Voting-based pose estimation for robotic assembly using a 3d sensor. In: 2012 IEEE International Conference on Robotics and Automation. IEEE, May 2012. https://doi.org/10.1109/icra.2012.6225371
Yan, W., Xu, Z., Zhou, X., Su, Q., Li, S., Wu, H.: Fast object pose estimation using adaptive threshold for bin-picking. IEEE Access 8, 63 055–63 064 (2020). https://doi.org/10.1109/access.2020.2983173
Leão, G., Costa, C.M., Sousa, A., Veiga, G.: Detecting and solving tube entanglement in bin picking operations. Appli. Sci. 10(7), 2264 (2020). https://doi.org/10.3390/app10072264
Lenz, I., Lee, H., Saxena, A.: Deep learning for detecting robotic grasps (2013). https://arxiv.org/abs/1301.3592
Mahler, J., et al.: Dex-net 1.0: A cloud-based network of 3d objects for robust grasp planning using a multi-armed bandit model with correlated rewards. In: 2016 IEEE International Conference on Robotics and Automation (ICRA). IEEE, May 2016. https://doi.org/10.1109/icra.2016.7487342
Mahler, J., et al.: Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics (2017). https://arxiv.org/abs/1703.09312
Zeng, A., et al.: Multi-view self-supervised deep learning for 6d pose estimation in the amazon picking challenge (2016). https://arxiv.org/abs/1609.09475
Le, T.-T., Lin, C.-Y.: Bin-picking for planar objects based on a deep learning network: A case study of USB packs. Sensors 19(16), 3602 (2019). https://doi.org/10.3390/s19163602
Höfer, T., Shamsafar, F., Benbarka, N., Zell, A.: Object detection and autoencoder-based 6d pose estimation for highly cluttered bin picking (2021). https://arxiv.org/abs/2106.08045
Danielczuk, M., et al.: Segmenting unknown 3d objects from real depth images using mask r-cnn trained on synthetic data (2018). https://arxiv.org/abs/1809.05825
Girshick, R.: Fast r-cnn (2015). https://arxiv.org/abs/1504.08083
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn (2017). https://arxiv.org/abs/1703.06870
Abdulla, W.: Mask r-cnn for object detection and instance segmentation on keras and tensorflow (2017). https://github.com/matterport/Mask_RCNN
Tested build configurations. https://www.tensorflow.org/install/source#gpu. (Accessed: 03 Mar 2022)
Mask r-cnn for object detection and segmentation using tensorflow 2.0. https://github.com/ahmedfgad/Mask-RCNN-TF2. (Accessed 13 Mar 2022)
Industrial 3d scanner: Phoxi®. https://www.photoneo.com/phoxi-3d-scanner/?gclid=EAIaIQobChMI_4y38LLp-AIVCcPVCh2E0A8eEAAYASAAEgJMwfD_BwE
Costa, C., Sobreira, H., Sousa, A., Veiga, G.: "Robust 3/6 dof self-localization system with selective map update for mobile robot platforms. Robot. Autonom. Syst. 76, 113–140 (2016)
de Souza Carvalho, J., et al.: Reconfigurable grasp planning pipeline with grasp synthesis and selection applied to picking operations in aerospace factories. Robot. Comput.-Integ. Manuf. 67, 07 (2020)
Acknowledgements
The research leading to these results has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 101006798.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cordeiro, A., Rocha, L.F., Costa, C., Silva, M.F. (2023). Object Segmentation for Bin Picking Using Deep Learning. In: Tardioli, D., Matellán, V., Heredia, G., Silva, M.F., Marques, L. (eds) ROBOT2022: Fifth Iberian Robotics Conference. ROBOT 2022. Lecture Notes in Networks and Systems, vol 590. Springer, Cham. https://doi.org/10.1007/978-3-031-21062-4_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-21062-4_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-21061-7
Online ISBN: 978-3-031-21062-4
eBook Packages: EngineeringEngineering (R0)