A biped static balance control and torque pattern learning under unknown periodic external forces

Abstract

This paper addresses a biped balancing task in which an unknown external force is exerted, using the so-called ‘ankle strategy’ model. When an external force is periodic, a human adaptively maintains the balance, next learns how much force should be produced at the ankle joint from its repeatability, and finally memorized it as a motion pattern. To acquire motion patterns with balancing, we propose a control and learning method: as the control method, we adopt ground reaction force feedback to cope with an uncertain external force, while, as the learning method, we introduce a motion pattern generator that memorizes the torque pattern of the ankle joint by use of Fourier series expansion. In this learning process, the period estimation of the external force is crucial; this estimation is achieved based on local autocorrelation of joint trajectories. Computer simulations and robot experiments show effective control and learning results with respect to unknown periodic external forces.

Introduction

A kind of intelligent motor behaviors of animals is observed in a learning of motion patterns by adapting to unknown environment. Even my pet dog, for instance, not only walks in my cluttered house without tripping but also changes his walking or running pattern according to the situation. Such behaviors are not easy to achieve as robot behaviors, because, in conventional robot controls, the motion patterns are programmed in advance by assuming environmental conditions. The programmed robot behaviors are not assured in unknown or variable environments.

Two types of abilities are found in the above dog behavior. One is an ability to learn motions as a special pattern appropriate to a steady environmental condition (walking is switched to running). The other is to stabilize the posture or movement to deal with a rapid disturbance from the environment (walking in the cluttered house where foot placement may be slightly different at each step). This paper treats a scheme of motor control and learning from these two points of view.

To make the problem as simple as possible, the balancing problem as shown in Fig. 1 is considered as an example. In this situation, a human stands on a floor where the slope changes periodically at a slow speed, e.g., on a large boat in the wild sea. Normally, humans can adjust their standing position with respect to the slope of the ground by changing the sway angle, in other words, the joint angle of the ankle. This balancing method is especially called ankle strategy (Horak and Nashner, 1986), although some other strategies, mechanisms or control methods are proposed in the human standing (Winter, 1995, Alexandrov et al., 1998, Rietdyk et al., 1999, Van Ooteghem et al., 2008) or biomechanical/robotic model (Gorce and Vanel, 1997, Hof, 2007, Mergner et al., 2009). In addition, if this situation continues for a long time, humans can learn the operation of the ankle as a periodic motion pattern based on this periodicity. The object of this paper is how to achieve such a balancing behavior by artificial machines like robots rather than to elucidate a neuronal mechanism whereby humans accomplish such an adaptive behavior, although the problem originates from a consideration of a control/learning mechanism in the behaviors of biological systems.

Balance control methods are often discussed in the field of robotics, especially for walking robots. Here, the main issue is rather motion generation based on the zero moment point (ZMP) (Vukobratovic et al., 1990), a point on the ground around which the moment of inertial force and gravity are balanced. In this method, reference trajectories of the joints or the body's center of gravity (CoG) are calculated in advance as a motion pattern so that the ZMP is kept beneath the foot support. Then the trajectories are reproduced by position control during actual walking (Takanishi et al., 1989, Hirai et al., 1998, Mitobe et al., 2001). On-line generation (Kajita and Tani, 1996, Nishiwaki et al., 2002, Sugihara et al., 2002, Behnke, 2006) or on-line modification (Huang et al., 2000, Wollherr and Buss, 2004, Lee et al., 2005, Prahlad et al., 2007) of the trajectories have been proposed to adapt to changes in environmental conditions. However, the discussions of periodic pattern learning are not yet sufficiently advanced to support the development of walking robots. As for static balance, the stability of the upright posture is analyzed (Napoleon and Sampei, 2002). The acquisition of biped dynamics is described with a neurophysiological model (Nakayama and Kimura, 2004). Reinforcement learning is applied (Borghese and Calvi, 2003), also to a stand-up behavior that requires a balancing task (Morimoto and Doya, 2001). Human static balance is measured to clarify its adaptive characteristics (Nashner, 1976, Priplata et al., 2002, Lockhart and Ting, 2007). Regarding to the periodic motion learning, a method based on the controllers containing oscillators, called by CPG, have been proposed (Nishii, 1999, Ishiguro et al., 2003, Ijspeert, 2008, Endo et al., 2008). However, control and pattern learning schemes of periodic motion based on the on-line balance have not been sufficiently discussed.

Thus, this paper deals with a problem such as that in Fig. 1 that contains both control and learning factors. We have already proposed a control and learning method for a special case where the period of an environmental alternation is given beforehand (Ito et al., 2005). In this paper, this method is extended to the case in which its period is also unknown. For this extension, estimation of the period is required. After the concept of our control scheme is explained in Section 2, a control and learning method including period estimation is formulated in Section 3. In Section 4, the effects of our scheme are confirmed by simulations, and it is applied to the motion control of actual robot in Section 5. In Section 6, concepts and assumptions in this paper are reconsidered and its advantages as well as remained problems are discussed. Finally, this paper is concluded in Section 7.