Abstract
Moving target tracking and grabbing is a common task for industrial robots. Usually, industrial robot complete complex actions through programming and teaching technologies, which suffers from the limitations of complicated programming logic and low scalability. Based on this, a flexible strategy combining deep learning and Kalman filter is proposed for eye-to-hand robotic tracking and grabbing. Firstly, the classic YOLOv3 algorithm is applied for target detection, and the bounding box of the target on the conveyor belt is obtained. Secondly, the target motion model is built up to obtain the system parameter matrices. Thirdly, the prediction equations can be given by Kalman filtering, and the target prediction position can be calculated and feedback to the robotic arm for the grabbing task. Finally, the experimental results show that the proposed strategy can improve the robustness of industrial robot tracking and grabbing, and its scalability is also improved compared with traditional methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Strandhagen, J.W., Alfnes, E., Strandhagen, J.O., Vallandingham, L.R.: The fit of Industry 4.0 applications in manufacturing logistics a multiple case study. Adv. Manuf. 5, 344–358 (2017). https://doi.org/10.1007/s40436-017-0200-y
Gao, M., Yan, Y., Yang, Y., Huang, J., He, Z.: A positioning system based on monocular vision for industrial robots. In: Proceedings of the 3rd International Conference on Information Science and Control Engineering (ICISCE 2016), Beijing, pp. 784–788 (2016). https://doi.org/10.1109/ICISCE.2016.172
Song, K., Chang, Y., Chen, J.: 3D vision for object grasp and obstacle avoidance of a collaborative robot. In: Proceedings of the IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM 2019), Hong Kong, China, pp. 254–258 (2019). https://doi.org/10.1109/AIM.2019.8868694
Zhang, J., Zhou, Z., Xing, L., Sheng, X., Wang, M.: Target recognition and Location based on deep learning. In: Proceedings of the IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC 2020), Chongqing, China, pp. 247–250 (2020). https://doi.org/10.1109/ITNEC48623.2020.9084826
Wang, S., Jiang, X., Zhao, J., Wang, X., Zhou, W., Liu, Y.: Vision based picking system for automatic express package dispatching. In: Proceedings of the IEEE International Conference on Realtime Computing and Robotics (RCAR 2019), Irkutsk, Russia, pp. 797–802 (2019). https://doi.org/10.1109/RCAR47638.2019.9044094
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015). https://doi.org/10.1038/nature14539
Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Girshick, R.: Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448. IEEE (2015)
Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement [EB/OL] (2018). https://arxiv.org/abs/1804.02767
Smeulders, A.W.M., Chu, D.M., Cucchiara, R., Calderara, S., Dehghan, A., Shah, M.: Visual tracking: an experimental survey. IEEE Trans. Pattern Anal. Mach. Intell. 36(7), 1442–1468 (2014). https://doi.org/10.1109/TPAMI.2013.230
Pallavi, S., Ramya Laxmi, K., Ramya, N., Raja, R.: Study and analysis of modified mean shift method and Kalman filter for moving object detection and tracking. In: Raju, K.S., Govardhan, A., Rani, B.P., Sridevi, R., Murty, M.R. (eds.) Proceedings of the Third International Conference on Computational Intelligence and Informatics. AISC, vol. 1090, pp. 821–828. Springer, Singapore (2020). https://doi.org/10.1007/978-981-15-1480-7_76
He, F., Zhen, J., Wang, Z.: Video target tracking based on adaptive Kalman filter. In: Liang, Q., Wang, W., Liu, X., Na, Z., Jia, M., Zhang, B. (eds.) Video Target Tracking Based on Adaptive Kalman Filter. LNEE, vol. 571, pp. 1198–1201. Springer, Singapore (2020). https://doi.org/10.1007/978-981-13-9409-6_141
Quigley, M., et al.: ROS: an open-source robot operating system. In: Proceedings of the ICRA Open-Source Software Workshop (2009)
Chitta, S.: MoveIt!: an introduction. In: Koubaa, A. (ed.) Robot Operating System (ROS). SCI, vol. 625, pp. 3–27. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-26054-9_1
Marchand, E., Spindler, F., Chaumette, F.: ViSP for visual servoing: a generic software platform with a wide class of robot control skills. IEEE Robot. Autom. Mag. 12(4), 40–52 (2005). https://doi.org/10.1109/MRA.2005.1577023
Qu, J., Zhang, F., Tang, Y., Fu, Y.: Dynamic visual tracking for robot manipulator using adaptive fading Kalman filter. IEEE Access 8, 35113–35126 (2020). https://doi.org/10.1109/ACCESS.2020.2973299
Wan, N., Yeung, D.: Learning a deep compact image representation for visual tracking. In: Proceedings of the 26th International Conference on Neural Information Processing Systems, pp. 809–817 (2013)
Acknowledgements
This work is supported by the National Natural Science Foundation of China (Grant number 61671194), Fundamental Research Funds for the Provincial Universities of Zhejiang (Grant number GK199900299012-010) and the Zhejiang Provincial Key Lab of Equipment Electronics (grant number 2019E10009).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhong, J., Sun, W., Cai, Q., Zhang, Z., Dong, Z., Gao, M. (2020). Deep Learning Based Strategy for Eye-to-Hand Robotic Tracking and Grabbing. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Lecture Notes in Computer Science(), vol 12533. Springer, Cham. https://doi.org/10.1007/978-3-030-63833-7_66
Download citation
DOI: https://doi.org/10.1007/978-3-030-63833-7_66
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63832-0
Online ISBN: 978-3-030-63833-7
eBook Packages: Computer ScienceComputer Science (R0)