Motor Skills Learning and Generalization with Adapted Curvilinear Gaussian Mixture Model

Abstract

This paper is intended to solve the motor skills learning, representation and generalization problems in robot imitation learning. To this end, we present an Adapted Curvilinear Gaussian Mixture Model (AdC-GMM), which is a general extension of the GMM. The proposed model can encode data more compactly. More critically, it is inherently suitable for representing data with strong non-linearity. To infer the parameters of this model, a Cross Entropy Optimization (CEO) algorithm is proposed, where the cross entropy loss of the training data is minimized. Compared with the traditional Expectation Maximization (EM) algorithm, the CEO can automatically infer the optimal number of components. Finally, the generalized trajectories are retrieved by an Adapted Curvilinear Gaussian Mixture Regression (AdC-GMR) model. To encode observations from different frames, the sophisticated task parameterization (TP) technique is introduced. All above proposed algorithms are verified by comprehensive tasks. The CEO is evaluated by a hand writing task. Another goal-directed reaching task is used to evaluate the AdC-GMM and AdC-GMR algorithm. A novel hammer-over-a-nail task is designed to verify the task parameterization technique. Experimental results demonstrate the proposed CEO is superior to the EM in terms of encoding accuracy and the AdC-GMM can achieve more compact representation by reducing the number of components by up to 50%. In addition, the trajectory retrieved by the AdC-GMR is smoother and the approximation error is comparable to the Gaussian process regression (GPR) even far fewer parameters need to be estimated. Because of this, the AdC-GMR is much faster than the GPR. Finally, simulation experiments on the hammer-over-a-nail task demonstrates the proposed methods can be deployed and used in real-world applications.

Appendices

Appendix

Proof of Equation (5)

Proof

According to Theorem 1 we have

$$\begin{array}{@{}rcl@{}} &&H^{\times} \left( \mu \Vert \mathcal{N}({\boldsymbol m}, {\boldsymbol {\Sigma}})\right) = H^{\times} \left( \mu_{\mathcal{G}} \Vert \mathcal{N}({\boldsymbol m}, {\boldsymbol {\Sigma}})\right) \\&&= H^{\times} \left( \mathcal{N}({\boldsymbol m}_{\mu}, {\boldsymbol {\Sigma}}_{\mu}) \Vert \mathcal{N}({\boldsymbol m}, {\boldsymbol {\Sigma}})\right) \end{array} $$

So the purpose is to prove

$$\begin{array}{@{}rcl@{}} &&{} H^{\times} \left( \mathcal{N}({\boldsymbol m}_{\mu}, {\boldsymbol {\Sigma}}_{\mu}) \Vert \mathcal{N}({\boldsymbol m}, {\boldsymbol {\Sigma}})\right) = \frac{N}{2}\ln (2\pi ) + \frac{1}{2}\text{tr}\left( {{{\boldsymbol {\Sigma}}^{- 1}}{{\boldsymbol {\Sigma}}_{\mu} }} \right) \\&&{} + \frac{1}{2}\ln \det \left( {\boldsymbol {\Sigma}} \right) + \frac{1}{2}\left\| {{\boldsymbol m} - {{\boldsymbol m}_{\mu} }} \right\|_{\boldsymbol {\Sigma}}^{2}. \end{array} $$

To prove this equation, we include another theorem, which declared that

Theorem 1

Let fbe a density function,$A:\mathbb {R}^{N} \rightarrow \mathbb {R}^{N}$isinvertible affine transform, define a transform$f_{A}: x \rightarrow f(A^{-1}x) / |\det A|$,then we have

$$ H^{\times} (\mu \circ {\boldsymbol A}^{-1} \Vert f_{\boldsymbol A}) =H^{\times} (\mu \Vert f) + \ln \left|\det {\boldsymbol A} \right| $$

(28)

where $\det A$ is the linear component of A. The proof is as follows

$$\begin{array}{@{}rcl@{}} && H^{\times} (\mu \circ {\boldsymbol A}^{-1} \Vert {f_{\boldsymbol A}})\\ &&= - \int \ln{\frac{f(\boldsymbol{A}^{-1} x)}{\left| \det \boldsymbol{A} \right|}} \mathrm{d} \mu \circ \boldsymbol{A}^{-1}(x) \\ &&= \ln \left| \det \boldsymbol{A} \right| - \int \ln{f(\boldsymbol{A}^{-1}x)} \mathrm{d} \mu \circ \boldsymbol{A}^{-1}(x) \\ && \overset{let y = \boldsymbol{A}^{-1}x}{\Longrightarrow} \ln \left| \det \boldsymbol{A} \right| - \int \ln{f(y)} \mathrm{d} \mu (y) \\ &&= H^{\times} (\mu \Vert f) + \ln \left|\det {\boldsymbol A} \right| \end{array} $$

□

Defining a transformation A : Σ^− 1(x − m) and apply it to both densities in Eq. 5. Let g and f be the transformed densities of μ and $\mathcal {N}(\boldsymbol {m, {\Sigma }})$, resptectively. Then g is a Gaussian with center m_g and covariance Σ_g and f is a standard normal distribution with zero mean and identity covariance I. Using Theorem 2, we have

$$ H^{\times} \left( \mathcal{N}({\boldsymbol m}_{\mu}, {\boldsymbol {\Sigma}}_{\mu}) \Vert \mathcal{N}({\boldsymbol m}, {\boldsymbol {\Sigma}})\right) = H^{\times} (g \Vert f) + \ln |\det {\Sigma}| $$

(29)

where,

$$\begin{array}{@{}rcl@{}} && H^{\times} (g \Vert f) = H^{\times} (g \Vert \mathcal{N}(0, I)) \\ &&= \int f_{g}(x) [\frac{N}{2} \ln(2\pi) + \frac{1}{2}\ln \det(I) + \frac{1}{2}||\boldsymbol{x}||^{2}] dx \quad\quad\quad\quad\quad\quad\text{Using definition (4)}\\ &&= \frac{N}{2} \ln(2\pi) + \frac{1}{2} \int f_{g}(x)|| (\boldsymbol{x} - \boldsymbol{m}_{g}) + \boldsymbol{m}_{g}||^{2} dx \\ &&= \frac{N}{2} \ln(2\pi) + \frac{1}{2} \int f_{g}(x) \left[|| \boldsymbol{(x - m_{g})}||^{2} + ||\boldsymbol{m}_{g}||^{2} + 2(\boldsymbol{x - m_{g}})^{T} \boldsymbol{m_{g}} \right] dx \\ &&\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad~~~~\mathbb{E}[c] = c, c \text{~is a constant}\\ &&= \frac{N}{2} \ln(2\pi) + \frac{1}{2}||\boldsymbol{m}_{g}||^{2} + \frac{1}{2} \mathbb{E}_{g} \left\{ tr[\boldsymbol{(x - m_{g})}\boldsymbol{(x - m_{g})}^{T}]\right\} \qquad~~~{tr(A) = tr(A^{T})} \\ &&= \frac{N}{2} \ln(2\pi) + \frac{1}{2}tr(\boldsymbol{\Sigma}_{g}) + \frac{1}{2}||\boldsymbol{m}_{g}||^{2} \end{array} $$

(30)

where ${\Sigma }_{g} = {\Sigma }^{-1} {\Sigma }_{\mu }$ and $||m_{g}||^{2} = ||m-m_{\mu }||_{\Sigma }^{2}$. Substituting (30) into Eq. 29 and plug in the value of m_g and Σ_g, we have (5) □