Robust Regression-Based Motion Perception for Online Imitation on Humanoid Robot

Abstract

Kinect is frequently used as a capture device for perceiving human motion in human–robot interaction. However, the Kinect’s principle of capture makes it possible for outliers to be present in the raw 3D joint position data, yielding an unsatisfying motion imitation by a humanoid robot. To eliminate these outliers and improve the precision of motion perception, we are inspired from the principle of signal restoration and propose a robust regression-based refining algorithm. We made contributions mainly in designing an Arc Tangent Square function to estimate the tendency of motion trajectories, and constructing a stepwise robust regression strategy to successively refine the outliers hidden in the motion capture data. The motion trajectories refined by the proposed algorithm are 40, 10, and 30% better than the raw motion capture data on spatial similarity, temporal similarity, and smoothness, respectively. In the online implementation on a humanoid robot NAO, the imitated motions of the human’s upper limbs are synchronous and accurate. The proposed robust regression-based refining algorithm realizes high-performance motion perception for online imitation of the humanoid robot.

Inverse Kinematics Transform (IKT) Formulas

In Appendix A, the left arm is taken as an example for providing the formulas of the IKT. The structure of the left arm is illustrated in Fig. 16. The left arm can be viewed as a two-link manipulator with four DOFs. Using the algebraic approach to solve the inverse kinematics problem can lead to a non-unique solution, while simultaneously having a considerable computational cost. Thus, the geometric solution approach, which is a hierarchical and intuitive approach, is used here to implement the IKT.

The shoulder joint angles are related to the poses of the torso and upper arm, while the elbow angles depend on the relative position of the upper and lower arms. Assuming that the position data of all of the joints has been converted into NAO coordinates, the x-axis of the torso coordinates is represented by the normal vector of the torso plane. This is constructed by the left shoulder ${{\varvec{P}}_{\mathrm {LS}}}$, the right shoulder ${{\varvec{P}}_{\mathrm {RS}}}$, and the center of the hip ${{\varvec{P}}_{\mathrm {CH}}}$. The vector from ${{\varvec{P}}_{\mathrm {RS}}}$ to ${{\varvec{P}}_{\mathrm {LS}}}$ denotes the y-axis, and the vector from ${{\varvec{P}}_{\mathrm {CH}}}$ to the center of shoulder ${{\varvec{P}}_{\mathrm {CS}}}$ denotes the z-axis. The rotation matrices ${{\varvec{R}}_{\mathrm {Torso}}}$ from NAO coordinates to the torso coordinates can then be obtained [27].

1.1 The IKT of the Shoulder Joint Angles

All the joint positions included in the left arm can be translated and rotated from NAO coordinates to the left shoulder coordinates. The left shoulder coordinates only have translation relationships (not rotational relationships) with the torso coordinates, and the joint positions can be presented as:

$$\begin{aligned} {{\varvec{P}}'_{\mathrm {LS}}}&= {\left( {{{x'}_\mathrm {LS}},{{y'}_\mathrm {LS}},{{z'}_\mathrm {LS}}} \right) ^\mathrm {T}} = {\varvec{O}}, \end{aligned}$$

(A.1)

$$\begin{aligned} {{\varvec{P}}'_{\mathrm {LE}}}&= {\left( {{{x'}_\mathrm {LE}},{{y'}_\mathrm {LE}},{{z'}_\mathrm {LE}}} \right) ^\mathrm {T}} = {{\varvec{R}}_{\mathrm {Torso}}}({{\varvec{P}}_{\mathrm {LE}}} - {{\varvec{P}}_{\mathrm {LS}}}), \end{aligned}$$

(A.2)

$$\begin{aligned} {{\varvec{P}}'_{\mathrm {LW}}}&= {\left( {{{x'}_\mathrm {LW}},{{y'}_\mathrm {LW}},{{z'}_\mathrm {LW}}} \right) ^\mathrm {T}} = {{\varvec{R}}_{\mathrm {Torso}}}({{\varvec{P}}_{\mathrm {LW}}} - {{\varvec{P}}_{\mathrm {LS}}}). \end{aligned}$$

(A.3)

where ${{\varvec{P}}_{\mathrm {LS}}}$, ${{\varvec{P}}_{\mathrm {LE}}}$, and ${{\varvec{P}}_{\mathrm {LW}}}$ are the joint positions in the NAO coordinates, while ${{\varvec{P}}'_{\mathrm {LS}}}$, ${{\varvec{P}}'_{\mathrm {LE}}}$, and ${{\varvec{P}}'_{\mathrm {LW}}}$ are the corresponding positions in the left shoulder coordinates. Equation (A.1) means that ${{\varvec{P}}'_{\mathrm {LS}}}$ is the origin of the left shoulder coordinates. According to Fig. 16a, ${\theta _{\mathrm {LSPitch}}}$ and ${\theta _{\mathrm {LSRoll}}}$ can be calculated by the following formulas:

$$\begin{aligned} {\theta _{\mathrm {LSPitch}}}&= \arctan \frac{{{{z'}_{\mathrm {LE}}} - {{z'}_{\mathrm {LS}}}}}{{{{x'}_{\mathrm {LE}}} - {{x'}_{\mathrm {LS}}}}} = \arctan \frac{{{{z'}_{\mathrm {LE}}}}}{{{{x'}_{\mathrm {LE}}}}}, \end{aligned}$$

(A.4)

$$\begin{aligned} {\theta _{\mathrm {LSRoll}}}&= \arctan \frac{{{{y'}_{\mathrm {LE}}} - {{y'}_{\mathrm {LS}}}}}{{\sqrt{{{({{x'}_{\mathrm {LE}}} - {{x'}_{\mathrm {LS}}})}^2} + {{({{z'}_{\mathrm {LE}}} - {{z'}_{\mathrm {LS}}})}^2}} }}\nonumber \\&= \arctan \frac{{{{y'}_{\mathrm {LE}}}}}{{\sqrt{{{x'}_{\mathrm {LE}}}^2 + {{z'}_{\mathrm {LE}}}^2} }}. \end{aligned}$$

(A.5)

Fig. 16

Sketch of the left arm. a Upper arm and the shoulder joint angles; b lower arm and the elbow joint angles

1.2 The IKT of the Elbow Joint Angles

The calculation of ${\theta _{\mathrm {LEYaw}}}$ and ${\theta _{\mathrm {LERoll}}}$ requires a further coordinate transform, i.e., from the left shoulder coordinates to the left elbow coordinates. The joint positions in the left elbow coordinates can be presented as follows:

$$\begin{aligned} {{\varvec{P}}''_\mathrm {LE}}&= {\left( {{{x''}_\mathrm {LE}},{{y''}_\mathrm {LE}},{{z''}_\mathrm {LE}}} \right) ^\mathrm {T}} = {\varvec{O}}, \end{aligned}$$

(A.6)

$$\begin{aligned} {{\varvec{P''}}_{\mathrm {LW}}}&= {\left( {{{x''}_\mathrm {LW}},{{y''}_\mathrm {LW}},{{z''}_\mathrm {LW}}} \right) ^\mathrm {T}}\nonumber \\&= {\varvec{R}}\left( z, - {\theta _{\mathrm {LSRoll}}} \right) {\varvec{R}} \left( y, - {\theta _{\mathrm {LSPitch}}} \right) ({{\varvec{P'}}_{\mathrm {LW}}} - {{\varvec{P'}}_{\mathrm {LE}}}), \end{aligned}$$

(A.7)

where ${\varvec{P}}''_{\mathrm {LE}}$ and ${\varvec{P}}''_{\mathrm {LW}}$ are the positions of the left elbow and wrist in the left elbow coordinates, respectively. Equation (A.7) indicates that the left shoulder coordinates first rotate $- {\theta _{\mathrm {LSPitch}}}$ around the y-axis, and then rotate $- {\theta _{\mathrm {LSRoll}}}$ around the z-axis. The expressions of ${\varvec{R}} \left( y, - {\theta _{\mathrm {LSPitch}}} \right)$ and ${\varvec{R}}\left( z, - {\theta _{\mathrm {LSRoll}}} \right)$ are:

$$\begin{aligned} {\varvec{R}} \left( y, - {\theta _{\mathrm {LSPitch}}} \right)&= \left[ {\begin{array}{*{20}{c}} {\cos ( - {\theta _{\mathrm {LSPitch}}})}&{}0&{}{\sin ( - {\theta _{\mathrm {LSPitch}}})}\\ 0&{}1&{}0\\ { - \sin ( - {\theta _{\mathrm {LSPitch}}})}&{}0&{}{\cos ( - {\theta _{\mathrm {LSPitch}}})} \end{array}} \right] , \end{aligned}$$

(A.8)

$$\begin{aligned} {\varvec{R}} \left( z, - {\theta _{\mathrm {LSRoll}}} \right)&= \left[ {\begin{array}{*{20}{c}} {\cos ( - {\theta _{\mathrm {LSRoll}}})}&{}{ - \sin ( - {\theta _{\mathrm {LSRoll}}})}&{}0\\ {\sin ( - {\theta _{\mathrm {LSRoll}}})}&{}{\cos ( - {\theta _{\mathrm {LSRoll}}})}&{}0\\ 0&{}0&{}1 \end{array}} \right] . \end{aligned}$$

(A.9)

According to Fig. 16b, the solutions of ${\theta _{\mathrm {LEYaw}}}$ and ${\theta _{\mathrm {LERoll}}}$ can be given as follows:

$$\begin{aligned} {\theta _{\mathrm {LEYaw}}}&= \arctan \frac{{{{z''}_{\mathrm {LW}}} - {{z''}_{\mathrm {LE}}}}}{{{{y''}_{\mathrm {LW}}} - {{y''}_{\mathrm {LE}}}}} = \arctan \frac{{{{z''}_{\mathrm {LW}}}}}{{{{y''}_{\mathrm {LW}}}}}, \end{aligned}$$

(A.10)

$$\begin{aligned} {\theta _{\mathrm {LERoll}}}&= \arctan \frac{{\sqrt{{{({{y''}_{\mathrm {LW}}} - {{y''}_{\mathrm {LE}}})}^2} + {{({{z''}_{\mathrm {LW}}} - {{z''}_{\mathrm {LE}}})}^2}} }}{{{{x''}_{\mathrm {LW}}} - {{x''}_{\mathrm {LE}}}}}\nonumber \\&= \arctan \frac{{\sqrt{{{y''}_{\mathrm {LW}}}^2 + {{z''}_{\mathrm {LW}}}^2} }}{{{{x''}_{\mathrm {LW}}}}}. \end{aligned}$$

(A.11)

The IKT formulas of the right arm can be derived in a similar fashion.