Learning to exploit passive compliance for energy-efficient gait generation on a compliant humanoid

Abstract

Modern humanoid robots include not only active compliance but also passive compliance. Apart from improved safety and dependability, availability of passive elements, such as springs, opens up new possibilities for improving the energy efficiency. With this in mind, this paper addresses the challenging open problem of exploiting the passive compliance for the purpose of energy efficient humanoid walking. To this end, we develop a method comprising two parts: an optimization part that finds an optimal vertical center-of-mass trajectory, and a walking pattern generator part that uses this trajectory to produce a dynamically-balanced gait. For the optimization part, we propose a reinforcement learning approach that dynamically evolves the policy parametrization during the learning process. By gradually increasing the representational power of the policy parametrization, it manages to find better policies in a faster and computationally efficient way. For the walking generator part, we develop a variable-center-of-mass-height ZMP-based bipedal walking pattern generator. The method is tested in real-world experiments with the bipedal robot COMAN and achieves a significant 18% reduction in the electric energy consumption by learning to efficiently use the passive compliance of the robot.

Appendix: bipedal walking gait generator

Given the z-axis CoM trajectory, we utilized the ZMP concept for x-axis and y-axis CoM trajectories, in order to obtain walking patterns with dynamic balance. To generate real-time bipedal walking patterns which use the vertical CoM trajectory generated by the RL component, we adopted the resolution method explained in Kagami et al. (2002), using Thomas Algorithm (Ugurlu et al. 2009). Considering the one mass model, CoM position and ZMP position are described as $P = (p_x, p_y, p_z)$ and $Q =(q_x, q_y, 0)$, respectively. As described in Kajita et al. (2003), Choi et al. (2007), Harada et al. (2004), Sugihara and Nakamura (2009), the abstracted x-axis ZMP equation takes the following form,

$$\begin{aligned} q_x = p_x - \frac{\ddot{p}_x}{\ddot{p}_z+g}p_z , \end{aligned}$$

(6)

where g is the gravitational acceleration. The vertical CoM position ($p_z$) and acceleration ($\ddot{p}_z$) are provided by the learning algorithm for all times as previously stated. As next step, (6) is discretized for $p_x$ as follows:

$$\begin{aligned} \ddot{p}_x(t) = \frac{p_x(i+1) - 2p_x(i) + p_x(i-1)}{\varDelta t^2}, \end{aligned}$$

(7)

where $\varDelta t$ is the sampling period, i is the discrete event. i starts from 0 to n which is the total number of discrete events. Inserting (7) into (6), we obtain the following:

$$\begin{aligned} p_x(i+1)= & {} \frac{b(i)}{c(i)}p_x(i) -p_x(i-1) +\frac{q_x(i)}{c(i)} ; \end{aligned}$$

(8)

$$\begin{aligned} b(i)= & {} 1 - 2c(i); \, \, \, \, \, \, \, \, \, \, \, \, c(i) = \frac{-p_z(i)}{(\ddot{p}_z(i)+g) \varDelta t^2} . \end{aligned}$$

(9)

In order to solve this tridiagonal equation efficiently, we employ Thomas Algorithm (Ugurlu et al. 2009). To do so, initial and final position of x-axis CoM ($p_x(0)$ and $p_x(n)$) must be given in advance. Therefore, for a given set of reference ZMP trajectory, initial conditions, and final conditions, we are able to calculate CoM trajectory. For that purpose, the tridiagonal equation is re-arranged as below.

$$\begin{aligned} {p_x}(i)=e(i+1)p_x(i+1)+f(i+1) . \end{aligned}$$

(10)

In (10), $e(i+1)$ and $f(i+1)$ can be defined as follows:

$$\begin{aligned} e(i+1)=-\frac{c(i)}{c(i)e(i)+b(i)} , \end{aligned}$$

(11)

$$\begin{aligned} f(i+1)=\frac{q_x(i)-c(i)f(i)}{c(i)e(i)+b(i)} . \end{aligned}$$

(12)

Combining (10), (11) and (12), (13) is yielded.

$$\begin{aligned} {p_x}(i)=-\frac{c(i)}{c(i)e(i)+b(i)}p_x(i+1)+\frac{q_x(i)-c(i)f(i)}{c(i)e(i)+b(i)} . \nonumber \\ \end{aligned}$$

(13)

Recall that $p_x(0) = x_0$ and $p_x(n) = x_n$, e(1) and f(1) are determined as 0 and $x_0$, respectively. Utilizing Thomas Algorithm for the solution of this tridiagonal equation, we can obtain the CoM trajectory’s x-axis component. If an identical approach is also executed for y-axis CoM position, we could derive all the components of the CoM trajectory in real-time since vertical CoM position is previously determined by the RL algorithm.