A Systematic Procedure for Comparing Template-Based Gesture Recognizers

Abstract

To consistently compare gesture recognizers under identical conditions, a systematic procedure for comparative testing should investigate how the number of templates, the number of sampling points, the number of fingers, and their configuration with other hand parameters such as hand joints, palm, and fingertips impact performance. This paper defines a systematic procedure for comparing recognizers using a series of test definitions, i.e. an ordered list of test cases with controlled variables common to all test cases. For each test case, its accuracy is measured by the recognition rate and its responsiveness by the execution time. This procedure is applied on six state-of-the-art template-based gesture recognizers on SHREC2019, a gesture dataset that contains simple and complex hand gestures tested and is largely used in the literature for competition in a user-independent scenario, and on Jackknife-lm, another challenging dataset. The results of the procedure identify the configurations in which each recognizer is the most accurate or the fastest.

Appendices

A The New Recognizers Considered in the Experiment

1.1 A.1 $P³+X Recognizer

A variant of $\textsf {\$}\textrm{P}^{3}+$ [27], that takes into account the direction-invariance by tracking conflicting templates (i.e., templates of the same gesture but performed in different directions). If a gesture matches with a conflicting template, its direction is compared with the direction of each conflicting template and the nearest one is chosen.

1.2 B.2 PennyPincher3D Recognizer

PennyPincher3D is an adaptation of the 2D recognizer PennyPincher [24]. The gestures are represented as a set of $N-1$ vectors linking between N equidistant points. The recognizer matches the candidate gesture with the template that maximizes a dissimilarity score, computed as the sum of the angles between the vectors. The computation relies on basic mathematical operations as additions and multiplications. The gestures require just a resampling as prepossessing. This recognizer is scale- and position- invariant as most of the $-recognizers.

B The Datasets Considered in the Experiment

1.1 C.3 SHREC2019 [10]

The SHREC2019 dataset [10] contains a sequence of 3D points and quaternions for each hand’s joint designating one of five gesture classes (Fig. 8): “Cross” (X), “Circle” (O), “V-mark” (V), “Caret” (/$\backslash$), and “Square” ([])). It served in (SHREC) track, a contest on online gesture recognition to detect command gestures from hands’ movements in a virtual reality context. The proposed dataset consists of 195 3D movements performed by 13 participants with the whole hand. The dataset contains unsegmented gestures, the training set and the testing set were merged to create a unique dataset in which, unnecessary hand movements were removed from the gestures.

Fig. 8.

The SHREC2019 gesture classes.

1.2 D.4 Jackknife-LM [25]

The Jackknife-LM (Jackknife-LeapMotion) dataset [25] contains 3D complex gestures of the hand and saved as 3D skeleton which is provided by the LeapMotion. We used the segmented gestures composed by 360 samples of 9 different gesture classes for example “Fist Circles”, “Snip Snip”, “Explode” (Fig. 9). It was used to test a rejection approach of non-gesture sequences from a continuous data stream. While segmented gestures make up the training set, authors employ unsegmented sessions of samples in the test [25].

Fig. 9.

The Jackknife-LM gesture classes [25].

C Recognition Rates

1.1 E.5 Recognition Rate for Best Condition for the SHREC2019

Fig. 10.

Recognition rate (left) and the number of recognized gestures per class over 100 trials (right) of all recognizers for user independent scenario, for the optimal conditions defined by the de Borda ranking: $A = 2\,(T + I)$, $N = 32$, $T = 16$. Error bars show a confidence interval of $95\%$.

1.2 F.6 Recognition Rates Tables for the SHREC2019

1.3 G.7 Recognition Rates Tables for the Jackknife-LM

1.4 H.8 The Ranking of the Recognizers Based on the Best Individual Rates by Articulation for SHREC2019

Fig. 11.

Ranking of the best individual recognition rates above 80% by number of joints (A) and number of points (N) for the SHREC2019 dataset.