Skip to main content
SpringerLink
Log in

A Learning Behavior Based Controller for Maintaining Balance in Robotic Locomotion

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

The Behavior Based Locomotion Controller (BBLC) extends the applicability of the behavior based control (BBC) architecture to redundant systems with multiple task-space motions. A set of control behaviors are attributed to each task-space motion individually and a reinforcement learning algorithm is used to select the combination of behaviors which can achieve the control objective. The resulting behavior combination is an emergent control behavior robust to unknown environments due to the added learning capability. Hence, the BBLC is applicable to complex redundant systems operating in unknown environments, where the emergent control behaviors can satisfy higher level control objectives such as balance in locomotion. The balance control problem of two robotic systems, a bipedal robot walker and a mobile manipulator, are used to study the performance of this controller. Results show that the BBLC strategy can generate emergent balancing strategies capable of adapting to new unknown disturbances from the environment, using only a small fixed library of balancing behaviors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Arkin, R.C.: Behavior-Based Robotics. MIT press (1998)

  2. Antonelli, G., Arrichiello, F., Chiaverini, S.: The Null-Space-based Behavioral control for autonomous robotic systems. Intel. Serv. Robotics. 1(1), 27–39 (2008)

    Article  Google Scholar 

  3. Arrichiello, F., Chiaverini, S., Indiveri, G., Pedone, P.: The Null-Space-based Behavioral Control for Mobile Robots with Velocity Actuator Saturations. Int. J. Robot. Res. 29(10), 1317–1337 (2010)

    Article  Google Scholar 

  4. Jayasiri, A., Mann, G.K.I., Gosine, R.G.: Behavior coordination of mobile robotics using supervisory control of fuzzy discrete event systems. EEE Trans. Syst. Man Cybern. B 41(5), 1224–38 (2011)

    Article  Google Scholar 

  5. Pettersson, O., Karlsson, L., Saffiotti, A.: Model-free execution monitoring in behavior-based robotics. IEEE Trans. Syst. Man Cybern. B 37, 890–901 (2007)

    Article  Google Scholar 

  6. Nakamura, Y: Advanced Robotics: Redundancy and Optimization. Addison-Wesley (1990)

  7. Vukobratovic, M., Juricic, D.: Contribution to the synthesis of biped gait. IEEE Trans. Biomed. Eng. 1, 1–6 (1969)

    Article  Google Scholar 

  8. Sugihara, T, Nakamura, Y.: Whole-body cooperative balancing of humanoid robot using COG Jacobian. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 2575–2580 (2002)

  9. Sugihara, T., Nakamura, Y., Inoue, H.: Real-time humanoid motion generation through ZMP manipulation based on inverted pendulum control. IEEE International Conference on Robotics and Automation, 1404–1409 (2002)

  10. Popovic, M.B., Goswami, A., Herr, H.: Ground Reference Points in Legged Locomotion : Definitions, Biological Trajectories and Control Implications. Int. J. Robot. Res. 24(12), 1013–1032 (2005)

    Article  Google Scholar 

  11. Qing, T., Rong, X., Jian, C.: Tip over avoidance control for biped robot. Robotica 27(06), 884–889 (2009)

    Article  Google Scholar 

  12. Pratt, J., Chew, C.-M., Torres, A., Dilworth, P., Pratt, G.: Virtual Model Control : An Intuitive Approach. Int. J. Robot. Res. 129(2) (2001)

  13. Hirai, K., Hirose, M., Haikawa, Y., Takenaka, T.: The development of Honda humanoid robot. IEEE International Conference on Robotics and Automation, 1321–1326 (1998)

  14. Hirose, M., Ogawa, K.: Honda humanoid robots development. Philosophical transactions. Series A, Mathematical, physical, and engineering sciences 365(1850), 11–9 (2007)

    Article  Google Scholar 

  15. Shih, C.L., Chiou, C.J.: The motion control of a statically stable biped robot on an uneven floor. IEEE Trans. Syst. Man Cybern. B 28(2), 244–9 (1998)

    Article  Google Scholar 

  16. Park, J.H., Kim, E.S.: Foot and body control of biped robots to walk on irregularly protruded uneven surfaces. IEEE Trans. Syst. Man Cybern. B 39(1), 289–97 (2009)

    Article  Google Scholar 

  17. Rebula, J., Canas, F., Pratt, J., Goswami, A.: Learning Capture Points for humanoid push recovery. IEEE-RAS International Conference on Humanoid Robots, 65–72 (2007)

  18. Stephens, B.: Humanoid push recovery. IEEE-RAS International Conference on Humanoid Robots, 589–595 (2007)

  19. Stephens, B.J., Atkeson, C.G.: Push Recovery by Stepping for Humanoid Robots with Force Controlled Joints. Primus, 52–59 (2010)

  20. Bortolami, S.B., DiZio, P., Rabin, E., Lackner, J.R.: Analysis of human postural responses to recoverable falls. Exp. Brain Res. 151(3), 387–404 (2003)

    Article  Google Scholar 

  21. Rebula, J.R., Canas, F., Pratt, J.E., Goswami, A.: Learning Capture Points for Bipedal Push Recovery. IEEE International Conference on Robotics and Automation, 1774–1774 (2008)

  22. Ferreira, J.P., Crisostomo, M., Coimbra, A.P.: Sagittal stability PD controllers for a biped robot using a neurofuzzy network and an SVR. Robotica 29(05), 717–731 (2010)

    Article  Google Scholar 

  23. Beranek, R., Fung, H., Ahmadi, M.: A Walking Stability Controller with Disturbance Rejection Based on CMP Criterion and Ground Reaction Force Feedback. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2011)

  24. Goswami, A., Kallem, V.: Rate of change of angular momentum and balance maintenance of biped robots. IEEE International Conference on Robotics and Automation, 3785–3790 (2004)

  25. Kajita, S., Kanehiro, F., Kaneko, K., Fujiwara, K., Harada, K., Yokoi, K., Hirukawa, H.: Resolved momentum control: Humanoid motion planning based on the linear and angular momentum. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 1644–1650 (2003)

  26. Popovic, M., Englehart: Angular Momentum Primitives for Human Walking: Biomechanics and Control. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 1685–1691 (2004)

  27. Zaier, R., Nagashima, F.: Motion Pattern Generator and Reflex System for Humanoid Robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 840–845 (2006)

  28. Zaier, R., Kanda, S.: Piecewise-Linear Pattern Generator and Reflex System for Humanoid Robots. In: IEEE International Conference on Robotics and Automation, pp 2188–2195 (2007)

  29. Zaier, R., Kanda, S.: Adaptive locomotion controller and reflex system for humanoid robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 2492–2497 (2008)

  30. Khoukhi, A.: Neural based RSPN multi-agent strategy for biped motion control. Robotica 19(06), 611–612 (2001)

    Article  Google Scholar 

  31. Sugano, S., Huang, Q., Kato, I.: Stability criteria in controlling mobile robotic systems. In: IEEE/RSJ International Conference on Intelligent Robots and Systems Proceedings, vol. 2, pp 832–838 (1993)

  32. Lee, B.: Real-time ZMP compensation method using null motion for mobile manipulators. IEEE International Conference on Robotics and Automation 2, 1967–1972 (2002)

    Google Scholar 

  33. Dubowsky, S., Vance, E.: Planning mobile manipulator motions considering vehicle dynamic stability constraints, pp 1271–1276 (1989)

  34. Hootsmans, N.A.M., Dubowsky, S.: Control of Mobile Manipulators Including Vehicle Dynamic Characteristics. In: Proceedings of the IV Topical Meeting on Robotics and Remote Systems (1991)

  35. Huang, Q.: Manipulator Motion Planning for Stabilizing a Mobile-Manipulator. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 467–472 (1995)

  36. Huang, Q., Sugano, S., Tanie, K.: Motion planning for a mobile manipulator considering stability and task constraints. In: IEEE International Conference on Robotics and Automation, vol. 3, pp 2192–2198 (1998)

  37. Antoska, V., Jovanović, K., Petrović, V.M., Baṡċarević, N., Stankovski, M.: Balance analysis of the mobile anthropomimetic robot under disturbances-zmp approach. International Journal of Advanced Robotic Systems 10(206) (2013)

  38. Rey, D., Papadoupoulos, E.: Online automatic tipover prevention for mobile manipulators. In: IEEE/RSJ International Conference on Intelligent Robot and Systems., vol. 3, pp 1273–1278 (1997)

  39. Alipour, K.: Moment-Height Tip-Over Measure for Stability Analysis of Mobile Robotic Systems. In: IEEE/RSJ International Conference on Intelligent Robots and Systems., pp 5546–5551 (2006)

  40. Ghaffari, A., Meghdari, A., Naderi, D., Eslami, S.: Tipover Stability Enhancement of Wheeled Mobile Manipulators Using an Adaptive Neuro- Fuzzy Inference Controller System. Eng. Technol., 241–247 (2008)

  41. Meghdari, a., Naderi, D., Alam, M.: Neural-network-based observer for real-time tipover estimation. Mechatronics 15(8), 989–1004 (2005)

    Article  Google Scholar 

  42. Papadopoulos, E.G., Rey, D.A.: A New Measure of Tipover Stability Margin for Mobile Manipulators. In: IEEE International Conference on Robotics and Automation (1996)

  43. Roan, P.R., Burmeister, A., Rahimi, A., Holz, K., Hooper, D.: Real-World Validation of Three Tipover Algorithms for Mobile Robots. In: IEEE International Conference on Robotics and Automation (2010)

  44. Hohn, O., Gerth, W.: Probabilistic Balance Monitoring for Bipedal Robots. Int. J. Robot. Res. 28(2), 245–256 (2009)

    Article  Google Scholar 

  45. Kalyanakrishnan, S., Goswami, A.: Learning To Predict Humanoid Fall. International Journal of Humanoid Robotics 08(02), 245 (2011)

    Article  Google Scholar 

  46. Sutton, R.S., Barto, G.A.: Reinforcement Learning: An Introduction. MIT press (1998)

  47. Watkins, C.J.C.H: Q-learning. Mach. Learn. 22(3–4), 25–292 (1992)

    Google Scholar 

  48. Peng, J., Williams, R.: Incremental multi-step Q-learning. Mach. Learn. 22(1), 283–290 (1996)

    Google Scholar 

  49. Luca, A.D., Oriolo, G., Samson, C.: Feedback Control of a Nonholonomic Car-like Robot, tech. rep., Laboratoire d’Analyse et d’Architecture des Systems, CNRS (1998)

  50. Barker, O., Beranek, R., Ahmadi, M.: Design of a 13 Degree-of-Freedom Biped Robot with a CAN-Based Distributed Digital Control System. In: IEEE International Conference on Advanced Intelligent Mechatronics, pp 836–841 (2010)

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical and Aerospace Engineering, Ottawa, Ontario, Canada

    Richard Beranek & Mojtaba Ahmadi

Authors
  1. Richard Beranek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mojtaba Ahmadi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Richard Beranek.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Beranek, R., Ahmadi, M. A Learning Behavior Based Controller for Maintaining Balance in Robotic Locomotion. J Intell Robot Syst 82, 189–205 (2016). https://doi.org/10.1007/s10846-015-0254-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-015-0254-7

Keywords

Search

Navigation