Recognizing interactions between human performers by ‘Dominating Pose Doublet’

Abstract

A graph theoretic approach is proposed to recognize interactions (e.g., handshaking, punching, etc.) between two human performers in a video. Pose descriptors corresponding to each performer in the video are generated and clustered to form initial codebooks of human poses. Compact codebooks of dominating poses for each of the two performers are created by ranking the poses of the initial codebooks using two different methods. First, an average centrality measure of graph connectivity is introduced where poses are nodes in the graph. The dominating poses are graph nodes sharing a close semantic relationship with all other pose nodes and hence are expected to be at the central part of the graph. Second, a novel similarity measure is introduced for ranking dominating poses. The ‘pose doublets’, all possible combinations of dominating poses of the two performers, are ranked using an improved centrality measure of a bipartite graph. The set of ‘dominating pose doublets’ that best represents the corresponding interaction are selected using the perceptual analysis technique. The recognition results on standard interaction datasets show the efficacy of the proposed approach compared to the state-of-the-art.

Appendix A: Deriving equation (10) [30]

In our problem of detecting meaningful ‘pose doublets’ for an interaction $A_n, \varDelta$ is the number of ‘pose doublets’ in $A_n$. Then, we can formulate the problem by a sequence of i.i.d. random variables $\{X_q\}_{q=1,2,3,\ldots ,\varDelta }$, such that $0\le X_q\le 1$. Let us define $X_q$ as,

$$\begin{aligned} X_q ={\left\{ \begin{array}{ll} 1 &{} \hbox {when} \; E(u,v)<\eta \; \hbox {for the `pose doublet'}\; I_q \\ 0 &{} \hbox {otherwise}, \end{array}\right. } \end{aligned}$$

(17)

for a given $\eta$, where $I_q$ is the qth ‘pose doublet’ in $A_n$. We set $S_{\varDelta }=\sum _{q=1}^{\varDelta }X_q$ ($S_{\varDelta }$ is the number of ‘pose doublets’ in $A_n$ having $E(u,v)$ less than $\eta$) and $\nu \varDelta =E\left[ S_{\varDelta }\right]$. Then for $\nu \varDelta < t < \varDelta$ (since $\nu$ is a probability value $<$1), putting $\sigma =\frac{t}{\varDelta }$ as in [5], according to Hoeffding’s inequality [30],

$$\begin{aligned} P_n^{\eta }=P(S_{\varDelta }\ge t)\le e^{-\varDelta \left( \sigma \log {\frac{\sigma }{\nu }}+(1-\sigma ) \log {\frac{1-\sigma }{1-\nu }}\right) }. \end{aligned}$$

(18)

In addition, the right hand term of this inequality satisfies

$$\begin{aligned} e^{-\varDelta \left( \sigma \log {\frac{\sigma }{\nu }}+(1-\sigma ) \log {\frac{1-\sigma }{1-\nu }}\right) }\le e^{-\varDelta (\sigma -\nu )^2H(\nu )}, \end{aligned}$$

(19)

where,

$$\begin{aligned} H(\nu ) ={\left\{ \begin{array}{ll}\frac{1}{1-2\nu }\log {\frac{1-\nu }{\nu }} &{} \hbox {when}\; 0<\nu <\frac{1}{2} \\ \frac{1}{2\nu (1-\nu )} &{} \hbox {when}\; \frac{1}{2}\le \nu <1 \end{array}\right. } \end{aligned}$$

(20)

This is Hoeffding’s inequality [30]. Hoeffding’s inequality is a popular theorem in Statistics. For proof of the theorem, the reader may refer to [30].

We then apply (18), (19) and (20) for finding the sufficient condition of $\epsilon$-meaningfulness. If we take $t$ satisfying both of the inequalities $t\ge \nu \varDelta +\sqrt{\frac{\log {\tau } - \log {\epsilon }}{H(\nu )}}\sqrt{\varDelta }$ and $t<\varDelta$, then using(18) and (19) and putting $\sigma =\frac{t}{\varDelta }$ we get

$$\begin{aligned} \varDelta (\sigma -\nu )^2 \ge \frac{\log {\tau }-\log {\epsilon }}{H(\nu )}. \end{aligned}$$

(21)

Then using (18) and (21), we get

$$\begin{aligned} P_n^{\eta }\le e^{-\varDelta (\sigma -\nu )^2H(\nu )}\le e^{-\log {\tau }+\log {\epsilon }}=\frac{\epsilon }{\tau }. \end{aligned}$$

(22)

This means by definition of meaningfulness, the cut-off $\eta$ is $\epsilon$-meaningful (according to Eq. (8)).

Since for $\nu$ in $(0,1), H(\nu )\ge 2$ (according to Eq. (20)) so from (21) we get the sufficient condition of meaningfulness as (10).