Abstract
Manipulating deformable liner objects, such as following a cable, is easy for human beings but presents a significant challenge for robots. Moreover, learning strategies in real world can bring damage to sensors and pose difficulties for data collection. In this paper, we propose a Reinforcement Learning method to generalize cable following skills from simulation to reality. The agent uses an end-to-end approach, directly inputting raw sensor data into the framework to generate robot actions. Meanwhile, a Sim-to-Real network is applied to enable the tactilemotor policy transfer. In particular, we use different perception modalities and representations as components of the observations and investigate how these factors impact cable following results. Our extensive experiments in simulation demonstrate that the success rate of cable following can be up to 81.85% when both visual and tactile features are put into the policy, compared to using only one type of modality. The proposed method provides valuable insights for deformable objects manipulating scenarios.
C. Sun and B. Duan—Contribute equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ha, K.H., Huh, H., Li, Z., Lu, N.: Soft capacitive pressure sensors: trends, challenges, and perspectives. ACS Nano 16(3), 3442–3448 (2022)
Lin, W., Wang, B., Peng, G., Shan, Y., Hu, H., Yang, Z.: Skin-inspired piezoelectric tactile sensor array with crosstalk-free row+ column electrodes for spatiotemporally distinguishing diverse stimuli. Adv. Sci. 8(3), 2002817 (2021). https://doi.org/10.1002/advs.202002817
Lepora, N.F., Lin, Y., Money-Coomes, B., Lloyd, J.: DigiTac: a digit-tactip hybrid tactile sensor for comparing low-cost high-resolution robot touch. IEEE Robot. Autom. Lett. 7(4), 9382–9388 (2022)
Yuan, W., Dong, S., Adelson, E.H.: GelSight: high-resolution robot tactile sensors for estimating geometry and force. Sensors 17(12), 2762 (2017). https://doi.org/10.3390/s17122762
Yan, M., Zhu, Y., Jin, N., Bohg, J.: Self-supervised learning of state estimation for manipulating deformable linear objects. IEEE Robot. Autom. Lett. 5(2), 2372–2379 (2020)
Pecyna, L., Dong, S., Luo, S.: Visual-tactile multimodality for following deformable linear objects using reinforcement learning. In: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3987–3994. IEEE, October 2022
Zhao, Y., Jing, X., Qian, K., Gomes, D.F., Luo, S.: Skill generalization of tubular object manipulation with tactile sensing and Sim2Real learning. Robot. Auton. Syst. 160, 104321 (2023). https://doi.org/10.1016/j.robot.2022.104321
Javdani, S., Tandon, S., Tang, J., O’Brien, J.F., Abbeel, P.: Modeling and perception of deformable one-dimensional objects. In 2011 IEEE International Conference on Robotics and Automation, pp. 1607–1614. IEEE, May 2011
Lui, W.H., Saxena, A.: Tangled: learning to untangle ropes with RGB-D perception. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 837–844. IEEE, November 2013
Khalifa, A., Palli, G.: New model-based manipulation technique for reshaping deformable linear objects. Int. J. Adv. Manuf. Technol., 1–9 (2022)
Yu, M., Zhong, H., Zhong, F., Li, X.: Adaptive control for robotic manipulation of deformable linear objects with offline and online learning of unknown models. arXiv preprint arXiv:2107.00194 (2021)
Hellman, R.B., Tekin, C., van der Schaar, M., Santos, V.J.: Functional contour-following via haptic perception and reinforcement learning. IEEE Trans. Haptics 11(1), 61–72 (2017)
She, Y., Wang, S., Dong, S., Sunil, N., Rodriguez, A., Adelson, E.: Cable manipulation with a tactile-reactive gripper. Int. J. Robot. Res. 40(12–14), 1385–1401 (2021)
Zhao, W., Queralta, J.P., Westerlund, T.: Sim-to-real transfer in deep reinforcement learning for robotics: a survey. In: 2020 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 737–744. IEEE, December 2020
Chebotar, Y., et al.: Closing the sim-to-real loop: adapting simulation randomization with real world experience. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 8973–8979. IEEE, May 2019
Andrychowicz, O.M., et al.: Learning dexterous in-hand manipulation. Int. J. Robot. Res. 39(1), 3–20 (2020)
Niu, H., Hu, J., Cui, Z., Zhang, Y.: Dr2l: Surfacing corner cases to robustify autonomous driving via domain randomization reinforcement learning. In: Proceedings of the 5th International Conference on Computer Science and Application Engineering, pp. 1–8, October 2021
James, S., et al.: Sim-to-real via sim-to-sim: data-efficient robotic grasping via randomized-to-canonical adaptation networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12627–12637 (2019)
Church, A., Lloyd, J., Lepora, N.F.: Tactile sim-to-real policy transfer via real-to-sim image translation. In: Conference on Robot Learning, pp. 1645–1654. PMLR, January 2022
Jianu, T., Gomes, D.F., Luo, S.: Reducing tactile Sim2Real domain gaps via deep texture generation networks. In: 2022 International Conference on Robotics and Automation (ICRA), pp. 8305–8311. IEEE, May 2022
Chen, W., et al.: Bidirectional sim-to-real transfer for GelSight tactile sensors with CycleGAN. IEEE Robot. Autom. Lett. 7(3), 6187–6194 (2022)
Todorov, E., Erez, T., Tassa, Y.: MuJoCo: a physics engine for model-based control. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5026–5033. IEEE, October 2012
Si, Z., Yuan, W.: Taxim: an example-based simulation model for GelSight tactile sensors. IEEE Robot. Autom. Lett. 7(2), 2361–2368 (2022)
Haarnoja, T., Zhou, A., Abbeel, P., Levine, S.: Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: International Conference on Machine Learning, pp. 1861–1870. PMLR, July 2018
Haarnoja, T., et al.: Soft actor-critic algorithms and applications. arXiv preprint arXiv:1812.05905 (2018)
Acknowledgments
This work is sponsored by the Natural Science Foundation of Jiangsu Province, China (No. BK20201264), Zhejiang Lab (No. 2022NB0AB02), and the National Natural Science Foundation of China (No. 61573101).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sun, C., Duan, B., Qian, K., Zhao, Y. (2023). Learning Tactilemotor Policy for Robotic Cable Following via Sim-to-Real Transfer. In: Yang, H., et al. Intelligent Robotics and Applications. ICIRA 2023. Lecture Notes in Computer Science(), vol 14271. Springer, Singapore. https://doi.org/10.1007/978-981-99-6495-6_6
Download citation
DOI: https://doi.org/10.1007/978-981-99-6495-6_6
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-6494-9
Online ISBN: 978-981-99-6495-6
eBook Packages: Computer ScienceComputer Science (R0)