Abstract
For systems with complex and unstable dynamics, such as humanoids, the use of model-based control within a hierarchical framework remains the tool of choice. This is due to the challenges associated with applying model-free reinforcement learning on such problems, such as sample inefficiency and limits on exploration of state space in the absence of safety/stability guarantees. However, relying purely on physics-based models comes with its own set of problems. For instance, the necessary limits on expressiveness imposed by committing to fixed basis functions, and consequently, their limited ability to learn from data gathered on-line. This gap between theoretical models and real-world dynamics gives rise to a need to incorporate a learning component at some level within the model-based control framework. In this work, we present a highly redundant wheeled inverted-pendulum humanoid as a testbed for experimental validation of some recent approaches proposed to deal with these fundamental issues in the field of robotics, such as: 1. Semi-parametric Gaussian Process-based approaches to computed-torque control of serial robots [1] 2. Probabilistic Differential Dynamic Programming framework for trajectory planning by high-level controllers [2, 3] 3. Barrier Certificate based safe-learning approaches for data collection to learn the dynamics of inherently unstable systems [4]. We discuss how a typical model-based hierarchical control framework can be extended to incorporate approaches for learning at various stages of control design and hierarchy, based on the aforementioned tools.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Nguyen-Tuong, D., Peters, J.: Using model knowledge for learning inverse dynamics. In: 2010 IEEE International Conference on Robotics and Automation (ICRA). IEEE (2010)
Pan, Y., Theodorou, E.: Probabilistic differential dynamic programming. In: Advances in Neural Information Processing Systems (2014)
Pan, Y., et al.: Prediction under uncertainty in sparse spectrum Gaussian processes with applications to filtering and control. International Conference on Machine Learning (2017)
Wang, L., Theodorou, E.A., Egerstedt, M.: Safe learning of quadrotor dynamics using barrier certificates. arXiv preprint arXiv:1710.05472 (2017)
Feng, S.: Online hierarchical optimization for humanoid control (2016)
Stilman, M., Olson, J., Gloss, W.: Golem krang: dynamically stable humanoid robot for mobile manipulation. In: 2010 IEEE International Conference on Robotics and Automation (ICRA). IEEE (2010)
Canete, L., Takahashi, T.: Disturbance compensation in pushing, pulling, and lifting for load transporting control of a wheeled inverted pendulum type assistant robot using the extended state observer. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE (2012)
Kane, T.R.: Dynamics of nonholonomic systems. J. Appl. Mech. 28(4), 574–578 (1961)
Featherstone, R.: Robot dynamics algorithms (1984)
Stilman, M., et al.: Robots using environment objects as tools the ‘MacGyver’ paradigm for mobile manipulation. In: 2014 IEEE International Conference on Robotics and Automation (ICRA). IEEE (2014)
Jacobson, D.H., Mayne, D.Q.: Differential dynamic programming (1970)
Tassa, Y., Erez, T., Todorov, E.: Synthesis and stabilization of complex behaviors through online trajectory optimization. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE (2012)
Chan, R.P.M., Stol, K.A., Halkyard, C.R.: Review of modelling and control of two-wheeled robots. Annu. Rev. Control 37(1), 89–103 (2013)
Nagarajan, U., Kantor, G., Hollis, R.L.: Trajectory planning and control of an underactuated dynamically stable single spherical wheeled mobile robot. In: IEEE International Conference on Robotics and Automation, ICRA 2009. IEEE (2009)
Nagarajan, U.: Dynamic constraint-based optimal shape trajectory planner for shape-accelerated underactuated balancing systems, pp. 27–31 (2010)
Zafar, M., Hutchinson, S., Theodorou, E.A.: Hierarchical Optimization for Whole-Body Control of Wheeled Inverted Pendulum Humanoids. arXiv preprint arXiv:1810.03074 (2018)
Zafar, M., Patel, A., Bogdan, V., Glaser, N., Aguillera, S., Hutchisnon, S.: Online center of mass estimation for a humanoid wheeled inverted pendulum robot. arXiv preprint arXiv:1810.03076 (2018)
Dantam, N., Stilman, M.: Robust and efficient communication for real-time multi-process robot software. In: 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids). IEEE (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zafar, M. et al. (2020). Semi-parametric Approaches to Learning in Model-Based Hierarchical Control of Complex Systems. In: Xiao, J., Kröger, T., Khatib, O. (eds) Proceedings of the 2018 International Symposium on Experimental Robotics. ISER 2018. Springer Proceedings in Advanced Robotics, vol 11. Springer, Cham. https://doi.org/10.1007/978-3-030-33950-0_34
Download citation
DOI: https://doi.org/10.1007/978-3-030-33950-0_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-33949-4
Online ISBN: 978-3-030-33950-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)