Skip to main content
SpringerLink
Log in

Trajectory Generation Using Dual-Robot Haptic Interface for Reinforcement Learning from Demonstration

  • Conference paper
  • First Online:
Robot 2023: Sixth Iberian Robotics Conference (ROBOT 2023)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 976))

Included in the following conference series:

  • 44 Accesses

Abstract

In learning robotics, techniques such as Learning from Demonstrations (LfD) and Reinforcement Learning (RL) have become widely popular among developers. However, this approximations can result in inefficient strategies when it comes to train more than one agent interacting in the same space with several objects and unknown obstacles. To solve this problematic, Reinforcement Learning from Demonstration (RLfD) allows the agent to learn and evaluate its performance from a set of demonstrations provided by a human expert while generalising from them using RL training. In dual-robot applications this approach is suitable for training agents that perform collaborative tasks. For this reason, a dual-robot haptic interface has been designed in order to produce dual manipulation trajectories to feed a RLfD agent. Haptics allows to perform high quality demonstrations following an impedance control approach. Trajectories obtained will be used as positive demonstrations so the training environment can generate automatic ones. As a result, this dual-robot haptic interface will provide a few trajectory demonstrations on dual manipulation in order to train agents using RL strategies. The aim of this research is to generate trajectories with this dual-robot haptic interface to train one or more agents following RLfD paradigms. Results show that trajectories performed with this interface present less error and deviation than others performed with a non-haptic interface, increasing the quality of the training data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Argall, B. D., Chernova, S., Veloso, M., Browning, B.: A survey of robot learning from demonstration. In: Robotics and Autonomous Systems, vol. 57, pp. 469-483. Elsevier (2009). https://doi.org/10.1016/j.robot.2008.10.024

  2. Calli, B., et al.: Yale-CMU-Berkeley dataset for robotic manipulation research. Int. J. Robot. Res. 36, 261–268. SAGE Publications Sage UK (2017). https://doi.org/10.1177/0278364917700714

  3. Chen, Q., Dallas, E., Shahverdi, P., Korneder, J., Rawashdeh, O. A., Geoffrey Louie, W. -Y.: A sample efficiency improved method via hierarchical reinforcement learning networks. In: 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 1498-1505. IEEE (2022). https://doi.org/10.1109/RO-MAN53752.2022.9900738

  4. Clark, J. P., Lentini, G., Barontini, F., Catalano, M. G., Bianchi, M., O’Malley, M. K.: On the role of wearable haptics for force feedback in teleimpedance control for dual-arm robotic teleoperation. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 5187-5193. IEEE (2019). https://doi.org/10.1109/ICRA.2019.8793652

  5. Corke, P. I.: A simple and systematic approach to assigning Denavit–Hartenberg parameters. IEEE Trans. Robot. 23, 590–594. IEEE (2007). https://doi.org/10.1109/TRO.2007.896765

  6. De Coninck, E., Verbelen, T., Van Molle, P., Simoens, P., Dhoedt, B.: Learning robots to grasp by demonstration. Robot. Auton. Syst. 127, 103474. Elsevier (2020). https://doi.org/10.1016/j.robot.2020.103474

  7. Gao, Q., Ju, Z., Chen, Y., Wang, Q., Zhao, Y., Lai, S.: Parallel dual-hand detection by using hand and body features for robot teleoperation. IEEE Trans. Hum.-Mach. Syst. 53(2), 417-426. IEEE (2023). https://doi.org/10.1109/THMS.2023.3243774

  8. Girbés-Juan, V., Schettino, V., Demiris, Y., Tornero, J.: Haptic and visual feedback assistance for dual-arm robot teleoperation in surface conditioning tasks. IEEE Trans. Haptics 14(1), 44–56. IEEE (2021). https://doi.org/10.1109/TOH.2020.3004388

  9. Girbés-Juan, V., Schettino, V., Gracia, L., Solanes, J.E., Demiris, Y., Tornero, J.: Combining haptics and inertial motion capture to enhance remote control of a dual-arm robot. J. Multimodal User Interfaces 16, 219–238 (2022). https://doi.org/10.1007/s12193-021-00386-8

    Article  Google Scholar 

  10. Hu, H., Zhao, Z., Yang, X., Lou, Y.: A Learning from Demonstration Method for Robotic Assembly with a Dual-Sub-6-DoF Parallel Robot. In: 2021 WRC Symposium on Advanced Robotics and Automation (WRC SARA), pp. 73-78. IEEE (2021). https://doi.org/10.1109/WRCSARA53879.2021.9612676

  11. Arulkumaran, K., Deisenroth, M.P., Brundage, M., Bharath, A.A.: Deep reinforcement learning: a brief survey. IEEE Signal Process. Mag. 34(6), 26–38. IEEE (2017). https://doi.org/10.1109/MSP.2017.2743240

  12. Kyrarini, M., Zheng, Q., Haseeb, M. A., Gräser, A.: Robot learning of assistive manipulation tasks by demonstration via head gesture-based interface. In: 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR), pp. 1139–1146. IEEE (2019). https://doi.org/10.1109/ICORR.2019.8779379

  13. Laghi, M., et al.: Shared-autonomy control for intuitive bimanual tele-manipulation. In: 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids), pp. 1-9. IEEE (2019). https://doi.org/10.1109/HUMANOIDS.2018.8625047

  14. Li, Y.: Deep reinforcement learning: an overview. In: arXiv preprint arXiv:1701.07274 (2017)

  15. Lindner, T., Milecki, A.: Reinforcement learning-based algorithm to avoid obstacles by the anthropomorphic robotic arm. Appl. Sci. (2022). https://doi.org/10.3390/app12136629

  16. Love, L.J., Book, W.J.: Force reflecting teleoperation with adaptive impedance control. In: IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 34, pp. 159–165. IEEE (2004). https://doi.org/10.1109/TSMCB.2003.811756

  17. Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., Abbeel P.: Overcoming exploration in reinforcement learning with demonstrations. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 6292–6299. Arxiv arXiv:1709.10089 (2018)

  18. Ramírez, J., Yu, W., Perrusquía, A.: Model-free reinforcement learning from expert demonstrations: a survey. Artif. Intell. Rev. 55, 3213–3241. Springer (2022). https://doi.org/10.1007/s10462-021-10085-1

  19. Ravichandar, H., Polydoros, A. S., Chernova, S., Billard, A.: Recent advances in robot learning from demonstration. In: Annual Review of Control, Robotics, and Autonomous Systems, pp. 297–330. Annual Reviews (2020). https://doi.org/10.1146/annurev-control-100819-063206

  20. Sasaki, M., Muguro, J., Kitano, F., Njeri, W., Matsushita, K.: Sim-real mapping of an image-based robot arm controller using deep reinforcement learning. Appl. Sci. (2022). https://doi.org/10.3390/app122010277

  21. Selvaggio, M., Abi-Farraj, F., Pacchierotti, C., Giordano, P. R., Siciliano, B.: Haptic-based shared-control methods for a dual-arm system. IEEE Robot. Autom. Lett. 3(4), 4249–4256. IEEE (2018). https://doi.org/10.1109/LRA.2018.2864353

  22. Si, W., Wang, N., Yang, C.: A review on manipulation skill acquisition through teleoperation-based learning from demonstration. Cogn. Comput. Syst. 3, 1–16. Wiley Online Library (2021). https://doi.org/10.1049/ccs2.12005

  23. Sun, X., Li, J., Kovalenko, A. V., Feng, W., Ou, Y.: Integrating reinforcement learning and learning from demonstrations to learn nonprehensile manipulation. IEEE Trans. Autom. Sci. Eng. 20(3), 1735–1744. IEEE (2023). https://doi.org/10.1109/TASE.2022.3185071

  24. Turlapati, S. H., Campolo, D.: Towards haptic-based dual-arm manipulation. Sensors 23, 376. MDPI (2022). https://doi.org/10.3390/s23010376

  25. Wu, R., Zhang, H., Peng, T., Fu, L., Zhao, J.: Variable impedance interaction and demonstration interface design based on measurement of arm muscle co-activation for demonstration learning. In: Biomedical Signal Processing and Control, pp. 8–18. Elsevier (2019). https://doi.org/10.1016/j.bspc.2019.02.008

  26. Xie, Z.W., Zhang, Q., Jiang, Z.N., Liu, H.: Robot learning from demonstration for path planning: a review. Sci. China Technol. Sci. 63(8), 1325–1334 (2020). https://doi.org/10.1007/s11431-020-1648-4

    Article  Google Scholar 

  27. Zhang, Y., Zhao, X., Tao, B., Ding, H.: Multi-objective synchronization control for dual-robot interactive cooperation using nonlinear model predictive policy. IEEE Trans. Ind. Electron. 70, 582–593. IEEE (2022). https://doi.org/10.1109/TIE.2022.3150090

  28. Zhang, S., Xia, Q., Chen, M., Cheng, S.: Multi-objective optimal trajectory planning for robotic arms using deep reinforcement learning. Sensors (2023). https://doi.org/10.3390/s23135974

  29. Zhao, W., Queralta, J. P., Westerlund, T.: Sim-to-real transfer in deep reinforcement learning for robotics: a survey. In: 2020 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 737–744, IEEE (2020). https://doi.org/10.1109/SSCI47803.2020.9308468

Download references

Acknowledgement

Research work was funded by the Spanish Government through the project PID2021-122685OB-I00.

Author information

Authors and Affiliations

  1. AUtomatics, RObotics, and Artificial Vision Lab, University Institute for Computer Research, University of Alicante, San Vicente Alicante, Spain

    Daniel Frau-Alfaro, Santiago T. Puente & Ignacio de Loyola Páez-Ubieta

  2. University Institute for Engineering Research, University Miguel Hernández, Elche, Spain

    Ignacio de Loyola Páez-Ubieta

Authors
  1. Daniel Frau-Alfaro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Santiago T. Puente
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ignacio de Loyola Páez-Ubieta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel Frau-Alfaro .

Editor information

Editors and Affiliations

  1. Dep. de Eng. Electrotecnica e de Computadores, University of Coimbra, Coimbra, Portugal

    Lino Marques

  2. Department of Electrónica Industrial, University of Minho, Escola de Engenharia, Guimarães, Portugal

    Cristina Santos

  3. Department of Electrical Engineering, Polytechnic Institute of Bragança, Bragança, Portugal

    José Luís Lima

  4. Centro Universitario de la Defensa, Zaragoza, Spain

    Danilo Tardioli

  5. Centre for Automation and Robotics UPM-CSIC, Universidad Politecnica de Madrid, Madrid, Spain

    Manuel Ferre

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Frau-Alfaro, D., Puente, S.T., Páez-Ubieta, I.d.L. (2024). Trajectory Generation Using Dual-Robot Haptic Interface for Reinforcement Learning from Demonstration. In: Marques, L., Santos, C., Lima, J.L., Tardioli, D., Ferre, M. (eds) Robot 2023: Sixth Iberian Robotics Conference. ROBOT 2023. Lecture Notes in Networks and Systems, vol 976. Springer, Cham. https://doi.org/10.1007/978-3-031-58676-7_36

Download citation

Publish with us

Policies and ethics

Search

Navigation