Abstract
Among the existing dual-arm cooperative grasping methods, the dual-arm cooperative grasping method based on RGB camera is the mainstream intelligent method. However, these methods often require predefined depth, difficult to adapt to changes in depth without modification. To solve this problem, this paper proposes a dual-arm cooperative grasping method based on RGB camera, which is suitable for scenes with variable depth, to increase the adaptability of dual-arm cooperation. Firstly, we build a mathematical model based on RGB camera, and use the markers attached to the target to obtain the depth information of the target. Then the 3D pose of the target under the robot world coordinate system is obtained by combining the depth information and pixel information. Finally, the task is assigned to the left and right robotic arms, and the target grabbing task is realized based on the main-auxiliary control. The proposed approach is validated in multiple experiments on a Baxter robot under different conditions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bai, W., et al.: Dual-arm coordinated manipulation for object twisting with human intelligence. In: 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 902–908. IEEE (2021)
Cai, J., Cheng, H., Zhang, Z., Su, J.: Metagrasp: data efficient grasping by affordance interpreter network. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 4960–4966. IEEE (2019)
Du, G., Wang, K., Lian, S., Zhao, K.: Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review. Artif. Intell. Rev. 54(3), 1677–1734 (2021)
Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F.J., Medina-Carnicer, R.: Generation of fiducial marker dictionaries using mixed integer linear programming. Pattern Recogn. 51, 481–491 (2016)
Griffin, B.A., Corso, J.J.: Learning object depth from camera motion and video object segmentation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12352, pp. 295–312. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58571-6_18
Ibarguren, A., Eimontaite, I., Outón, J.L., Fletcher, S.: Dual arm co-manipulation architecture with enhanced human-robot communication for large part manipulation. Sensors 20(21), 6151 (2020)
Jocher, G., et al.: ultralytics/yolov5: v5. 0-yolov5-p6 1280 models aws supervise. ly and youtube integrations. Zenodo 11 (2021)
Laghi, M., et al.: Shared-autonomy control for intuitive bimanual tele-manipulation. In: 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids), pp. 1–9. IEEE (2018)
Liang, J., Mahler, J., Laskey, M., Li, P., Goldberg, K.: Using DVRK teleoperation to facilitate deep learning of automation tasks for an industrial robot. In: 2017 13th IEEE Conference on Automation Science and Engineering (CASE), pp. 1–8. IEEE (2017)
Lipton, J.I., Fay, A.J., Rus, D.: Baxter’s homunculus: virtual reality spaces for teleoperation in manufacturing. IEEE Robot. Autom. Lett. 3(1), 179–186 (2017)
Liu, D., et al.: A novel and efficient distance detection based on monocular images for grasp and handover. In: Gao, H., Wang, X. (eds.) CollaborateCom 2021. LNICST, vol. 406, pp. 642–658. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-92635-9_37
Lundell, J., Verdoja, F., Kyrki, V.: Beyond top-grasps through scene completion. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 545–551. IEEE (2020)
Medjram, S., Brethe, J.F., Benali, K.: Markerless vision-based one cardboard box grasping using dual arm robot. Multimedia Tools Appl. 79(31), 22617–22633 (2020)
Muñoz-Salinas, R., Marín-Jimenez, M.J., Yeguas-Bolivar, E., Medina-Carnicer, R.: Mapping and localization from planar markers. Pattern Recogn. 73, 158–171 (2018)
Ott, C., Nakamura, Y.: Employing wave variables for coordinated control of robots with distributed control architecture. In: 2008 IEEE International Conference on Robotics and Automation, pp. 575–582. IEEE (2008)
Punlum, V., Srisertpol, J., Khaengkam, S.: The application of double arms scara robot for deburring of PCB support plate. In: 2017 International Conference on Circuits, Devices and Systems (ICCDS), pp. 1–5. IEEE (2017)
Rastegarpanah, A., Marturi, N., Stolkin, R.: Autonomous vision-guided bi-manual grasping and manipulation. In: 2017 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO), pp. 1–7. IEEE (2017)
Romero-Ramirez, F.J., Muñoz-Salinas, R., Medina-Carnicer, R.: Speeded up detection of squared fiducial markers. Image Vis. Comput. 76, 38–47 (2018)
SepúLveda, D., Fernández, R., Navas, E., Armada, M., González-De-Santos, P.: Robotic aubergine harvesting using dual-arm manipulation. IEEE Access 8, 121889–121904 (2020)
Silvério, J., Clivaz, G., Calinon, S.: A laser-based dual-arm system for precise control of collaborative robots. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 9183–9189. IEEE (2021)
Smith, C., et al.: Dual arm manipulation-a survey. Robot. Auton. Syst. 60(10), 1340–1353 (2012)
Tung, A., et al.: Learning multi-arm manipulation through collaborative teleoperation. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 9212–9219. IEEE (2021)
Wu, Q., Li, M., Qi, X., Hu, Y., Li, B., Zhang, J.: Coordinated control of a dual-arm robot for surgical instrument sorting tasks. Robot. Auton. Syst. 112, 1–12 (2019)
Yang, Y., Liu, Y., Liang, H., Lou, X., Choi, C.: Attribute-based robotic grasping with one-grasp adaptation. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 6357–6363. IEEE (2021)
Yu, X., Zhang, S., Sun, L., Wang, Y., Xue, C., Li, B.: Cooperative control of dual-arm robots in different human-robot collaborative tasks. Assem. Autom. 40(1), 95–104 (2019)
Zahavi, A., Haeri, S.N., Liyanage, D.C., Tamre, M.: A dual-arm robot for collaborative vision-based object classification. In: 2020 17th Biennial Baltic Electronics Conference (BEC), pp. 1–5. IEEE (2020)
Zhong, F., Wang, Y., Wang, Z., Liu, Y.H.: Dual-arm robotic needle insertion with active tissue deformation for autonomous suturing. IEEE Robot. Autom. Lett. 4(3), 2669–2676 (2019)
Acknowledgement
This work was supported by the Project of NSFC (Grant No. U1908214), Special Project of Central Government Guiding Local Science and Technology Development (Grant No. 2021JH6/10500140), the Program for Innovative Research Team in University of Liaoning Province (LT2020015), the Support Plan for Key Field Innovation Team of Dalian (2021RT06), the Science and Technology Innovation Fund of Dalian (Grant No. 2020JJ25CY001), the Support Plan for Leading Innovation Team of Dalian University (Grant No. XLJ202010) and Dalian University Scientific Research Platform Project (No. 202101YB03).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Zhang, H., Yi, P., Liu, R., Dong, J., Zhang, Q., Zhou, D. (2022). Research on Depth-Adaptive Dual-Arm Collaborative Grasping Method. In: Gao, H., Wang, X., Wei, W., Dagiuklas, T. (eds) Collaborative Computing: Networking, Applications and Worksharing. CollaborateCom 2022. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 461. Springer, Cham. https://doi.org/10.1007/978-3-031-24386-8_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-24386-8_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-24385-1
Online ISBN: 978-3-031-24386-8
eBook Packages: Computer ScienceComputer Science (R0)