Cross-view action matching using a novel projective invariant on non-coplanar space-time points

Abstract

Existing action matching methods from the geometric respect typically assume the collinearity or coplanarity for view invariance. These assumptions curb the application to uncontrolled action patterns. In this paper, a new projective invariant named characteristic number (CN) is used, which can be used to describe 3D non-coplanar points. For motion trajectories of actions, we propose the temporal CN (TCN) for individual joint point of a human body in temporal series. This view-invariant feature can characterize an action well with limited number of joints(a single one in our experiments). In addition to TCN, we are also able to define the spatial characteristic number (SCN) on several (five in our paper) joint points in the spatial domain for one frame. SCN works complementary to temporal features, when limited snapshots of an action are available. We validate both SCN and TCN on the widely used CMU Motion Capture Database (Mocap) database, KTH Multiview Football Dataset II and IXMAS dataset. The promising recognition results indicate the invariance to varying viewpoints compared with the state-of-the-art. The results on CMU and KTH database corrupted by noise show the robustness to noise.

Appendix: Proof of Theorem 1

Assume A is the matrix of a given projective transformation Φ. $\mathcal {P}^{\prime }=\{P_{i}^{\prime }\}_{i=1}^{r}$ represent the projected points of $\mathcal {P}=\{P_{i}\}_{i=1}^{r}$ with Φ, and P _i ′=Φ(P _i )=k _i A P _i (P _r+1 = P ₁, k _i+1 = k ₁). $\mathcal {Q}=\left \{Q_{i}^{(j)}\right \}_{i=1,2,\ldots ,r}^{j=1,2,\ldots ,n}$ are projected to $\mathcal {Q}^{\prime }=\left \{Q_{i}^{(j)\prime }\right \}_{i=1,2,\ldots ,r}^{j=1,2,\ldots ,n}$, then

$$\begin{array}{@{}rcl@{}} Q_{i}^{(j)\prime}&=&\varPhi(Q_{i}^{(j)})= l_{i}^{(j)}AQ_{i}^{(j)} = l_{i}^{(j)}A \cdot (a_{i}^{(j)} P_{i} + b_{i}^{(j)} P_{i+1}) \\ &=&\frac{l_{i}^{(j)}a_{i}^{(j)}}{k_{i}} \cdot k_{i}AP_{i} + \frac{l_{i}^{(j)}b_{i}^{(j)}}{k_{i+1}} \cdot k_{i+1}AP_{i+1} \\ &=&\frac{l_{i}^{(j)}a_{i}^{(j)}}{k_{i}}P_{i}^{\prime} + \frac{l_{i}^{(j)}b_{i}^{(j)}}{k_{i+1}} P_{i+1}^{\prime}. \end{array} $$

Thus, the characteristic number of transformed points is given by

$$\begin{array}{@{}rcl@{}} CN(\mathcal{P}^{\prime},\mathcal{Q}^{\prime})&=& \prod\limits_{i=1}^{r} \prod\limits_{j=1}^{n} \left( \frac{k_{i+1}l_{i}^{(j)}a_{i}^{(j)}}{k_{i}l_{i}^{(j)}b_{i}^{(j)}}\right) \\ &=& \prod\limits_{i=1}^{r} \frac{k_{i+1}}{k_{i}} \left( \prod\limits_{j=1}^{n}\frac{a_{i}^{(j)}}{b_{i}^{(j)}}\right) \\ &=& \prod\limits_{i=1}^{r} \left( \prod\limits_{j=1}^{n}\frac{a_{i}^{(j)}}{b_{i}^{(j)}}\right) = CN(\mathcal{P},\mathcal{Q}), \end{array} $$

which indicates that the characteristic number is invariant under projective transformations.