Abstract
Grasping from a random pile is a great challenging application for robots. Most deep reinforcement learning-based methods focus on grasping of a single object. This paper proposes a novel structure for robot grasping from a pile with deep Q-learning, where each robot action is determined by the result of its current step and the next n steps. In the learning structure, a convolution neural network is employed to extract the target position, and a full connection network is applied to calculate the Q value of the grasping action. The former network is a pre-trained network and the latter one is a critical network structure. Moreover, we deal with the “reality gap” from the deep Q-learning policy learned in simulated environments to the real-world by large-scale simulation data and small-scale real data.
This work was supported in part by NSFC under Grant number 91848109, supported by Beijing Natural Science Foundation under Grant number L201019, and major scientific and technological innovation projects in Shandong Province Grant number 2019JZZY010430.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Goins, A.K., Carpenter, R., Wong, W.K., Balasubramanian, R.: Evaluating the efficacy of grasp metrics for utilization in a gaussian process-based grasp predictor. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3353–3360 (2014). https://doi.org/10.1109/IROS.2014.6943029
Boularias, A., Kroemer, O., Peters, J.: Learning robot grasping from 3-d images with markov random fields. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1548–1553 (2011). https://doi.org/10.1109/IROS.2011.6094888
Balaguer, B., Carpin, S.: Learning end-effector orientations for novel object grasping tasks. In: 2010 10th IEEE-RAS International Conference on Humanoid Robots, pp. 302–307 (2010). https://doi.org/10.1109/ICHR.2010.5686826
Saxena, A., Driemeyer, J., Ng, A.Y.: Robotic grasping of novel objects using vision. Int. J. Rob. Res. 27(2), 157–173 (2008)
Bohg, J., Morales, A., Asfour, T., Kragic, D.: Data-driven grasp synthesis-a survey. IEEE Trans. Rob. 30(2), 289–309 (2013)
Pinto, L., Gupta, A.: Supersizing self-supervision: Learning to grasp from 50k tries and 700 robot hours. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 3406–3413. IEEE (2016)
Mahler, J., et al.: Learning ambidextrous robot grasping policies. Sci. Rob. 4(26) (2019)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Silver, D., et al.: Mastering the game of go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)
Gu, S., Holly, E., Lillicrap, T., Levine, S.: Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 3389–3396. IEEE (2017)
Nair, A., Pong, V., Dalal, M., Bahl, S., Lin, S., Levine, S.: Visual reinforcement learning with imagined goals. arXiv preprint arXiv:1807.04742 (2018)
Pinto, L., Andrychowicz, M., Welinder, P., Zaremba, W., Abbeel, P.: Asymmetric actor critic for image-based robot learning. arXiv preprint arXiv:1710.06542 (2017)
Popov, I., Heess, N., Lillicrap, T., Hafner, R., Barth-Maron, G., Vecerik, M., Lampe, T., Tassa, Y., Erez, T., Riedmiller, M.: Data-efficient deep reinforcement learning for dexterous manipulation. arXiv preprint arXiv:1704.03073 (2017)
Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J., Quillen, D.: Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. The International Journal of Robotics Research 37(4–5), 421–436 (2018)
Sadeghi, F., Toshev, A., Jang, E., Levine, S.: Sim2real view invariant visual servoing by recurrent control. arXiv preprint arXiv:1712.07642 (2017)
Chen, B., Su, J.: Addressing reward engineering for deep reinforcement learning on multi-stage task. In: Gedeon, T., Wong, K.W., Lee, M. (eds.) ICONIP 2019. CCIS, vol. 1143, pp. 309–317. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-36802-9_33
Zeng, A., Song, S., Welker, S., Lee, J., Rodriguez, A., Funkhouser, T.: Learning synergies between pushing and grasping with self-supervised deep reinforcement learning. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4238–4245. IEEE (2018)
Wiering, M., Van Otterlo, M.: Reinforcement learning. Adapt. Learn. Optim. 12(3) (2012)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
James, S., et al.: Sim-to-real via sim-to-sim: Data-efficient robotic grasping via randomized-to-canonical adaptation networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12627–12637 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, B., Su, J., Wang, L., Gu, Q. (2021). Learning Robot Grasping from a Random Pile with Deep Q-Learning. In: Liu, XJ., Nie, Z., Yu, J., Xie, F., Song, R. (eds) Intelligent Robotics and Applications. ICIRA 2021. Lecture Notes in Computer Science(), vol 13014. Springer, Cham. https://doi.org/10.1007/978-3-030-89098-8_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-89098-8_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-89097-1
Online ISBN: 978-3-030-89098-8
eBook Packages: Computer ScienceComputer Science (R0)