Sparse pose manifolds

Abstract

The efficient manipulation of randomly placed objects relies on the accurate estimation of their 6 DoF geometrical configuration. In this paper we tackle this issue by following the intuitive idea that different objects, viewed from the same perspective, should share identical poses and, moreover, these should be efficiently projected onto a well-defined and highly distinguishable subspace. This hypothesis is formulated here by the introduction of pose manifolds relying on a bunch-based structure that incorporates unsupervised clustering of the abstracted visual cues and encapsulates appearance and geometrical properties of the objects. The resulting pose manifolds represent the displacements among any of the extracted bunch points and the two foci of an ellipse fitted over the members of the bunch-based structure. We post-process the established pose manifolds via $l_1$ norm minimization so as to build sparse and highly representative input vectors that are characterized by large discrimination capabilities. While other approaches for robot grasping build high dimensional input vectors, thus increasing the complexity of the system, in contrast, our method establishes highly distinguishable manifolds of low dimensionality. This paper represents the first integrated research endeavor in formulating sparse pose manifolds, with experimental results providing evidence of low generalization error, justifying thus our theoretical claims.

Appendix

1.1 Formulation of bunch-based architecture

Given an image of an object with certain pose, we first extract the 2D locations of the $\rho$ SIFT keypoints denoted as $\mathbf v _{\varvec{\zeta }}\in \mathbb {R}^2$. Then we perform clustering over the locations of the extracted interest points (input vectors $\mathbf v _{\varvec{\zeta }},\, \zeta =1,2,\dots ,\rho$) in order to account for the topological attributes of the object. Supposing there are $\gamma$ clusters denoted as $\mathbf b _\mathbf k ,\,k=1,2,\dots \gamma$, we consider basic Bayesian rules noting that a vector $\mathbf v _{\varvec{\zeta }}$ belongs to a cluster $\mathbf b _\mathbf k$ if $P(\mathbf b _\mathbf k |\mathbf v _{\varvec{\zeta }})>P(\mathbf b \varvec{_\zeta }|\mathbf v _\mathbf j ),\,\zeta ,k=1,2,\dots ,\gamma ,\,\zeta \ne j$. The expectation of the unknown parameters conditioned on the current estimates ${\varvec{\varTheta }} (\tau )$ ($\tau$ denotes the iteration step) and the training samples (E-step of the EM algorithm) are:

$$\begin{aligned} J({\varvec{\varTheta }};&{\varvec{\varTheta }}(\tau ))=E\Big [ \sum _{i=1}^{\rho }ln(p(\mathbf v _{\varvec{\zeta }}|\mathbf b _\mathbf k ;\varvec{\theta })P_\mathbf k ) \Big ] \nonumber \\&=\sum _{\zeta =1}^{\rho }\sum _{k=1}^{\gamma }P(\mathbf b _\mathbf k |\mathbf v _{\varvec{\zeta }}; {\varvec{\varTheta }}(\tau ))ln(p(\mathbf v _{\varvec{\zeta }}|\mathbf b _\mathbf k ;\varvec{\theta })P_\mathbf k ) \end{aligned}$$

(6)

with $\mathbf P _{1\times \gamma }=[P_1,\dots ,P_\gamma ]^T$ denoting the a priori probability of the respective clusters, $\widehat{\varvec{\theta }_{2\times \gamma }}=[\varvec{\theta }_1^T, \dots , \varvec{\theta }\gamma ^T]^T$ corresponding to the $\varvec{\theta _k}$ vector of parameters for the $k-th$ cluster and ${\varvec{\varTheta }}_{3\times \gamma }=[\hat{\varvec{\theta }^T},\mathbf P ^T]^T$. According to M-step of the EM algorithm, the parameters of the $\gamma$ clusters in the respective subspace are estimated through the maximization of the expectation:

$$\begin{aligned} {\varvec{\varTheta }}(\tau +1)=\arg \max _{\varvec{\varTheta }}J({\varvec{\varTheta }};{\varvec{\varTheta }}(\tau )) \end{aligned}$$

(7)

resulting in

$$\begin{aligned} \sum _{\zeta =1}^{\rho }\sum _{k=1}^{\gamma }P(\mathbf b _\mathbf k |\mathbf v _{\varvec{\zeta }};{\varvec{\varTheta }}(\tau ))\frac{\partial }{\partial \varvec{\theta }_\mathbf k }ln(p(\mathbf v _{\varvec{\zeta }}|\mathbf b _\mathbf k ;\varvec{\theta }_\mathbf k ))=0 \end{aligned}$$

(8)

while maximization with respect to the a priori probability P gives:

$$\begin{aligned} P_k=\frac{1}{\rho }\sum _{\zeta =1}^{\rho }P(\mathbf{b}_\mathbf{k}|\mathbf{v}_{\varvec{\zeta }};{\varvec{\varTheta }}(\tau ))\qquad \quad \text { with } k=1,\dots ,\gamma \end{aligned}$$

(9)

It is apparent that the optimization of Eq. 8 with respect to P stands for a constraint maximization problem that has to obey to $P_k\ge 0,\,k=1,\dots ,\gamma$ and $\sum _{k=1}^{\gamma }P_k=1$. We revise the Lagrangian theory that states:

Given a function $f(x)$ to be optimized subject to several constraints built the corresponding Lagrangian function as $\mathcal {L}(x,\lambda )=f(x)-\sum \lambda f(x)$.

Following on from the above statement, we denote the respective (to Eq. 6) Lagrangian function as:

$$\begin{aligned} \mathcal {J}(\mathbf P ,\lambda )=J({\varvec{\varTheta }};{\varvec{\varTheta }}(\tau ))-\lambda \Big (\sum _{k=1}^{\gamma }P_k-1\Big ) \end{aligned}$$

We obtain $\lambda$ and $P_k$ though:

$$\begin{aligned}&\partial \frac{\mathcal {J}(\mathbf P ,\lambda )}{\partial P_k}=0 \Rightarrow \\&\partial \frac{\Big ( \sum _{\zeta =1}^{\rho }\sum _{k=1}^{\gamma }P(\mathbf b _\mathbf k |\mathbf v _{\zeta }; {\varvec{\varTheta }}(\tau ))ln(p(\mathbf v _{\zeta }|\mathbf b _\mathbf k ;\varvec{\theta })P_\mathbf k )\Big )}{\partial P_k}-\\&-\frac{\lambda (\sum _{k=1}^{\gamma }P_k-1}{\partial P_k}=0 \Rightarrow \\&P_k=\frac{1}{\lambda }\sum _{\zeta =1}^{\lambda }P(\mathbf b _\mathbf k |\mathbf v _{\zeta };{\varvec{\varTheta }}(\tau )) \\&\text {Since } \sum _{k=1}^{\gamma }P_k=1, \text {we can derive that } \lambda =\rho \\&\text {resulting in the final } a priori \text { probability of the } k-th \\&\text {cluster of Eq. 9:}\\&P_k=\frac{1}{\rho }\sum _{\zeta =1}^{\rho }P(\mathbf b _\mathbf k |\mathbf v _{\zeta };{\varvec{\varTheta }}(\tau ))\qquad \qquad \text { with } k=1,\dots ,\gamma \end{aligned}$$

1.2 Training with noise

The performance of the proposed regressor-based 3D pose estimation module is bootstrapped by adding noise to the input vectors fed to the RBF-kernel during training. In the following passage we present a slightly modified version of the Tikhonov regularization theorem as adjusted to the needs of our case. In cases where the inputs do not contain noise and the size $\mathbf t$ of the training dataset tends to infinity, the error function containing the joint distributions $p(\mathbf y \varvec{_\lambda },\mathbf r )$ (of the desired values for the network output $\mathbf g \varvec{_\lambda }$) assumes the form:

$$\begin{aligned} E&=\lim _{\mathbf{t} \rightarrow \infty }\frac{1}{2\mathbf{t}}\sum _{k=1}^{\mathbf{t}}\sum _{\varvec{\lambda }}\{\mathbf{g}\varvec{_\lambda }(\mathbf{r}_\mathbf{k};\mathbf{w})-\mathbf{y}\varvec{_\lambda }_{\mathbf{k}}\}^2\\&=\frac{1}{2}\sum _{m}\int \int \{\mathbf{g}\varvec{_\lambda }(\mathbf{r}_\mathbf{k};\mathbf{w})-\mathbf{y}\varvec{_\lambda }_{\mathbf{k}}\}^2p(\mathbf{y}\varvec{_\lambda },\mathbf{r})d\mathbf{y}\varvec{_\lambda } d\mathbf{r}\\&=\frac{1}{2}\sum _{m}\int \int \{\mathbf{g}\varvec{_\lambda }(\mathbf{r}_\mathbf{k};\mathbf{w})-\mathbf{y}\varvec{_\lambda }_{\mathbf{k}}\}^2p(\mathbf{y}\varvec{_\lambda }|\mathbf{r})p(\mathbf{r})d\mathbf{y}\varvec{_\lambda } d\mathbf{r} \end{aligned}$$

Let $\varvec{\eta }$ be a random vector describing the input data with probability distribution $p(\varvec{\eta })$. In most of the cases, noise distribution is chosen to have zero mean ($\int \varvec{\eta }_ip(\varvec{\eta })d\varvec{\eta }=0$) and to be uncorrelated ($\int \varvec{\eta }_i\varvec{\eta }_jp(\varvec{\eta })d\varvec{\eta }=\text {variance}\sigma _{ij}$). In cases where each input data point contains additional noise and is repeated infinite times, the error function over the expanded data can be written as:

$$\begin{aligned} \widetilde{E}&=\frac{1}{2}\sum _{m}\int \int \int \{\mathbf{g}\varvec{_\lambda }((\mathbf{r}_\mathbf{t};\mathbf{w})+ \varvec{\eta })-\mathbf{y}\varvec{_\lambda }_{\mathbf{k}}\}^2 \\&p(\mathbf{y}\varvec{_\lambda } \mid \mathbf{r})p(\mathbf{r})p(\varvec{\eta })d\mathbf{y}\varvec{_\lambda } d\mathbf{r}d\varvec{\eta } \end{aligned}$$

Expanding the network function as a Taylor series in powers of $\eta$ produces:

$$\begin{aligned} \mathbf{g}\varvec{_\lambda }((\mathbf{r}_\mathbf{t};\mathbf{w})+&\varvec{\eta })=\mathbf{g}\varvec{_\lambda }(\mathbf{r}_\mathbf{t};\mathbf{w})+\sum _{i}\varvec{\eta }_i\frac{\partial \mathbf{g}\varvec{_{\lambda }}}{\partial \mathbf{r}_i}\biggm \vert _{\varvec{\eta }=0}+ \\&+\frac{1}{2}\sum _{i}\sum _{j}\varvec{\eta }_i\varvec{\eta }_j\frac{\partial ^2\mathbf{g}\varvec{_{\lambda }}}{\partial \mathbf{r}_i \partial \mathbf{r}_j}\biggm \vert _{\varvec{\eta }=0}+\mathcal {O}(\varvec{\eta }^3) \end{aligned}$$

By substituting the Taylor series expansion into the error function we obtain the following form of regularization term that governs the Tikhonov regularization:

$$\begin{aligned} \widetilde{E}=E+variance \times \varOmega \end{aligned}$$

with

$$\begin{aligned} \varOmega =\frac{1}{2}\sum _{m}\sum _{i}\int \int&\left\{ (\frac{\partial \mathbf{g}\varvec{_\lambda }}{\partial \mathbf{r}_i})^2 +\frac{1}{2}\{\mathbf{g}\varvec{_\lambda }(\mathbf{r})-\mathbf{y}\varvec{_\lambda }\}\frac{\partial ^2\mathbf{g}\varvec{_\lambda }}{\partial \mathbf{r}_{i}^{2}} \right\} \\&p(\mathbf{y}\varvec{_\lambda }|\mathbf{r})p(\mathbf{r})d\mathbf{r}d\mathbf{y}\varvec{_\lambda } \end{aligned}$$