Abstract
In this paper, we study a vision-based reactive adaptation method for contact-rich manipulation tasks, based on the compliant control and learning from demonstration. For contact-rich tasks, the compliant control method is essential, especially when interacting with a deformable object with unknown properties, such as pizza dough. Learning from demonstration (LfD) provides a solution for this challenging task. However, the generalisation capabilities of LfD for deformable object manipulation tasks are still a challenging and opening issue, especially for unknown and dynamic tasks. Therefore, in this work, we investigate the vision and force-based perception feedback to enhance the generalisation of the LfD outcomes. The computer vision algorithm was developed to recognise the shape of the object and calculate the deviation between the desired shape and the current shape. The deviation of shape adjusts the parameters of learned primitive skills encoded by Dynamic Movement Primitives (DMPs). We adopt the pizza dough rolling task as the typical case to evaluate the performance of the proposed method. The shape and thickness of the dough can be made to the desired shape and thickness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Note nonprehensile manipulation means manipulation without grasping, such as pushing, flipping, throwing, and squeezing.
- 2.
Note we ignore the friction force in practice.
- 3.
Note the pizza dough is not as large as the real pizza dough, because the roller is not large enough. The proposed method can be employed for big dough if we adopt a large roller.
References
Si, W., Wang, N., Yang, C.: A review on manipulation skill acquisition through teleoperation-based learning from demonstration. Cognit. Comput. Syst. 3(1), 1–16 (2021)
Matas, J., James, S., Davison, A.J.: Sim-to-real reinforcement learning for deformable object manipulation. In: Conference on Robot Learning. PMLR, pp. 734–743 (2018)
Hu, Z., Sun, P., Pan, J.: Three-dimensional deformable object manipulation using fast online gaussian process regression. IEEE Robot. Automat. Lett. 3(2), 979–986 (2018)
Si, W., Guan, Y., Wang, N.: Adaptive compliant skill learning for contact-rich manipulation with human in the loop. IEEE Robot. Automat. Lett. 7(3), 5834–5841 (2022)
Kim, J.-T., Ruggiero, F., Lippiello, V., Siciliano, B.: Planning framework for robotic pizza dough stretching with a rolling pin. In: Robot Dynamic Manipulation. Springer, pp. 229–253 (2022)
Gutiérrez-Giles, A., Ruggiero, F., Lippiello, V., Siciliano, B.: Closed-loop control of a nonprehensile manipulation system inspired by the pizza-peel mechanism. In: 18th European Control Conference (ECC). IEEE 2019, pp. 1580–1585 (2019)
Petit, A., Lippiello, V., Fontanelli, G.A., Siciliano, B.: Tracking elastic deformable objects with an rgb-d sensor for a pizza chef robot. Robot. Auton. Syst. 88, 187–201 (2017)
Satici, A.C., Ruggiero, F., Lippiello, V., Siciliano, B.: A coordinate-free framework for robotic pizza tossing and catching. In: 2016 IEEE International Conference on Robotics and Automation (ICRA). IEEE, pp. 3932–3939 (2016)
Si, W., Wang, N., Li, Q., Yang, C.: A framework for composite layup skill learning and generalizing through teleoperation. Front. Neurorobotics 16 (2022)
Lu, Z., Wang, N., Yang, C.: A novel iterative identification based on the optimised topology for common state monitoring in wireless sensor networks. Int. J. Syst. Sci. 53(1), 25–39 (2022)
Figueroa, N., Ureche, A.L.P., Billard, A.: Learning complex sequential tasks from demonstration: a pizza dough rolling case study. In: 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, pp. 611–612 (2016)
Espiau, B., Chaumette, F., Rives, P.: A new approach to visual servoing in robotics. IEEE Trans. Robot. Autom. 8(3), 313–326 (1992)
Pairet, È., Ardón, P., Mistry, M., Petillot, Y.: Learning generalizable coupling terms for obstacle avoidance via low-dimensional geometric descriptors. IEEE Robot. Automat. Lett. 4(4), 3979–3986 (2019)
Oktay, O., Schlemper, J., Folgoc, L.L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N.Y., Kainz, B. et al.: Attention u-net: learning where to look for the pancreas. arXiv preprint arXiv:1804.03999 (2018)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, pp. 234–241 (2015)
Mezirow, J.: Perspective transformation. Adult Educ. 28(2), 100–110 (1978)
Hartley, R., Zisserman, A.: Multiple view Geometry in Computer Vision, 2nd edn. Cambridge University Press, New York, NY, USA (2003)
Ijspeert, A.J., Nakanishi, J., Hoffmann, H., Pastor, P., Schaal, S.: Dynamical movement primitives: learning attractor models for motor behaviors. Neural Comput. 25(2), 328–373 (2013)
Si, W., Wang, N., Yang, C.: Composite dynamic movement primitives based on neural networks for human-robot skill transfer. Neural Comput. Appl. 1–11 (2021)
Billard, A., Mirrazavi, S., Figueroa, N.: Learning for Adaptive and Reactive Robot Control: A Dynamical Systems Approach. MIT Press (2022)
Ude, A., Nemec, B., Petrić, T., Morimoto, J.: Orientation in cartesian space dynamic movement primitives. In: 2014 IEEE International Conference on Robotics and Automation (ICRA). IEEE, pp. 2997–3004 (2014)
Saveriano, M., Piater, J.: e & i Elektrotechnik und Informationstechnik 137(6), 309–315 (2020). https://doi.org/10.1007/s00502-020-00816-7
Hogan, N.: Impedance control: an approach to manipulation: Part i-theory (1985)
Acknowledgements
This work was supported in part by the H2020 Marie Skłodowska-Curie Actions Individual Fellowship under Grant 101030691.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Si, W., Guo, C., Dong, J., Lu, Z., Yang, C. (2023). Deformation-Aware Contact-Rich Manipulation Skills Learning and Compliant Control. In: Borja, P., Della Santina, C., Peternel, L., Torta, E. (eds) Human-Friendly Robotics 2022. HFR 2022. Springer Proceedings in Advanced Robotics, vol 26. Springer, Cham. https://doi.org/10.1007/978-3-031-22731-8_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-22731-8_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-22730-1
Online ISBN: 978-3-031-22731-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)