1 Introduction

With the increase of cross-disciplinary research interest in development in exergames and coaching systems technology, it is expected that end-users will soon benefit from advancements aimed to improve/regain kinaesthetic awareness, skills and technique of motion patterns that require balance, fluidity and control. Coaching proprioception and kinaesthetic awareness by incorporating stance of the body with specific movements has a potential to advance sport performance, rehabilitation and related areas promoting an active life and to reach broad audience including busy, aging or less-physically active population. In tennis, stance coaching foundations (as body position during the swing) is aimed at improving performance and safety associated with balance, agility, movement fluidity and reaction times. The objective of this study is to quantify and extract the stance angle of body position in tennis groundstrokes from captured 3D motion data without ball impact information. It is expected that extraction of body orientation information during recognised activity pattern is transferrable to future exergames, augmented coaching systems (ACS) and other disciplines including surveillance, robot and automotive vision.

The following sections cover exergames and tennis domain-specific backgrounds related to stance. The mid-section includes 3D motion capture, data set analysis, developed algorithms with intermediate results and customisation tasks visualisation. The last section provides discussion, critique, results and concluding remarks.

1.1 Motion Capture, Exergames and Augmented Coaching Systems

Increasing popularity in exergames and 3D-based augmented coaching technology is attributed to various factors such as: advancements in computer vision (CV), open source CV, multi-camera DLT systems, and various ICT infrastructures with available SDKs that can capture and process human motion. From a pioneering Nintendo’s hand-held motion capture device to the advanced concept of Microsft’s Kinect using the body as a controller, the research community has benefited with available development platforms with SDKs for low-cost motion data capture. In this study, it is contended that variable accuracy of Xbox 360 Kinect depth mapping 0.4−8 m [1] and sampling frequency at 30 Hz, will improve with the next generations of exergames. Although the main advancement of the second generation of the Kinect sensor [2] may be perceived as having improved video resolution, the next generation of motion capture devices are expected to capture larger volumes, higher frequency, accuracy and precision of human motion data to allow further research advancements in coaching capabilities of exergames and early prototypes of ACS [3–6].

At present, to the best of our knowledge there are no available exergames that are able to reward performance of player’s solid base of support or assess body stance.

1.2 Tennis and Sport Science Backgrounds

Stance is a concept associated with body position during the swing action relative to the intended target line. In coaching practice, stance variations are typically introduced as a player progresses beyond beginner or advanced-beginner level. The basic stance, known as ‘side-on’ or ‘square stance’ is empirically considered in coaching practice as the most favourable body position to help novice players to find their optimal swing power and stable body position. Historically, in the era of wooden racquets and predominant single-hand backhands, players were generally thought to execute their groundstrokes from square stances while today’s forehands and predominant two-hand backhands are generally executed from all four common stances, mostly from semi-open and open stances [7–11].

To gain a competitive advantage, players of all skill levels aim to optimise and personalise stance selection associated with weight transfer, impact speed, and ball rebound height as a response to the opponent’s actions. For example, the ‘square’ stance (or slightly ‘closed’ stance approach to the ball) towards the target line generally offers more stability relative to the ‘open’ stance at ball impact. Furthermore, hitting the ball on the run along the baseline with e.g. ‘closed stance’ makes it harder to accelerate the racquet before impact, compared to the optimal stance for a player when hitting ‘winners’ [8, 12]. On the other hand, the potentially less-safe, ‘open stance’, allows extended court coverage and fast response to opponents actions at the expense of balance and produced impact velocity of the racquet on the ball [7, 13].

In coaching practice there is a known phenomenon of disagreements over decision boundaries (e.g. ‘is it square or closed stance?’) or prioritising feedback and intervention [14]. Given the subjective and qualitative analytical nature of coaching, this study provides methods and algorithms that could quantify i.e. extract the feet angle from a groundstroke stance and determine the groundstroke category (i.e. forehand or backhand) from captured 3D motion data for exergames and ACS.

2 Experimental Data Collection, Annotation and Analysis

The tennis motion data set was recorded at 50 Hz by nine video cameras from fixed locations. The data set contained 3D coordinates of a set of retro-reflective markers (q = 22) attached to anatomical body landmarks and a racquet (ASCII text data exported from SMART-e 900 eMotion/BTS). A rigid topology of interconnected marker’s trajectories was used to represent an avatar for an animated interactive 3D stick figure player. The tennis experts and end-users were able to visually assess diverse hitting stances (Figs. 1 and 2) with no recorded contact with the ball. The motion data set contained forehand and backend tennis swings with a balanced distribution of relatively good and bad swings that were considered as common errors [14].

Fig. 1.
figure 1

Visualising stance relative to the target line

Full size image
Fig. 2.
figure 2

Data set analysis: Stance distribution and expert-based classification into four output labels (0 – square stance, 1 – semi-open, 2 – open and 3 – closed stance).

Full size image

Stance assessment heuristics and utilised concept:

  • Extracting the stance angle from a hitting position is possible by comparing the imaginary line connecting the tips of the tennis shoes and the target line direction.

  • The extracted stance angle must have only one value associated with the swing type. Therefore, as a concept, for a right handed player, the front leg for forehands is the left leg, while for backhands it is the right leg, regardless of the player’s stance.

The captured multi-time-series of motion data are denoted as:

  • Human motion is represented via 3D marker position trajectories \( m_{p} (t) = (x_{p} (t),y_{p} (t),z_{p} (t)) \) in the captured volume.

  • Marker’s projections into x, y and z axis are denoted as: \( \overrightarrow {Xm} ,\overrightarrow {Ym} ,\overrightarrow {Zm} \,.\,\; \)

  • Each motion data sample is a frame of constituent markers M = (m 1 , m 2 ,…, m q ).

  • A frame M, representing a static stick-figure (shown in Fig. 3) is an interconnected finite set of three-dimensional marker positions in Euclidean space.

    Fig. 3.
    figure 3

    Constructed 3D stick figure in right-handed Euclidean space

    Full size image

  • Each swing event S j is a set of k consecutive frames: S j  = (t,…, t + k; M 1,… ,q ), or \( S_{j} = \left\{ {M(t)|t_{0} \le t \le t_{k - 1} } \right\} \), where \( m_{p} \, \in M;\;p = 1, \ldots ,q;\,\;i,j,k,p,q \in \) ℕ.

3 Methods and Experimental Results

Swing type and stance angle extraction are considered as temporal and spatial composition of computational steps. The presented algorithms are optimised to work with a minimalistic subset of markers (Fig. 3). As such, the hip markers PSGT and SSGT (great trochanter) are used as the equivalent concept of front leg heuristic by computing the front and rear hip position for stance angle extraction. Although the captured data contained only right-handed swing actions, the presented approach is based on the hand holding a racquet (i.e. PSHD marker).

3.1 Swing Type Recognition and Stance Angle Extraction

Before computing a stance angle, the output of the swing type algorithm (Table 1) is used to determine front and rear hip sides of the pelvis in relation to swing type.

Table 1. Pseudo code for computing swing type recognition.
Full size table

Before the stance angle extraction within the swing S j event, the swing type algorithm (Table 1) includes determining a leading hip for forehand and backhand from the existing markers’ time-series and relative pelvis centre (greater sciatic notch). Derived markers (in Eqs. (1) and (2)) are labelled as near_rear_hipM and x–z plane body centre i.e. computed pelvis centre PelvisC_GT. The near_rear_hipM is assigned after the swing type recognition (Table 1) to initialise stance angle algorithm (Table 2).

$$ near\_rear\_hipM = \left\{ {\begin{array}{*{20}c} {PSGT\therefore near\_front\_hipM = SSGT,if(swing\_{\text{type}} = FOREHAND)} \\ {SSGT\therefore near\_front\_hipM = PSGT,if(swing\_{\text{type}} = BACKHAND)} \\ \end{array} } \right. $$
(1)
$$ \begin{aligned} \overrightarrow {Xbody\_centre} = \frac{{\left| {\overrightarrow {Xssgt} - \overrightarrow {Xpsgt} } \right|}}{2} + \overrightarrow {Xnear\_rear\_hipMarker} \, \hfill \\ \overrightarrow {Ybody\_centre} = \frac{{\left| {\overrightarrow {Yssgt} - \overrightarrow {Ypsgt} } \right|}}{2} + \overrightarrow {Ynear\_rear\_hipMarker} \, \hfill \\ \overrightarrow {Zbody\_centre} = \frac{{\left| {\overrightarrow {Zssgt} - \overrightarrow {Zpsgt} } \right|}}{2} + \overrightarrow {Znear\_rear\_hipMarker} \, \hfill \\ \end{aligned} $$
(2)
Table 2. Pseudo code for static and dynamic stance as body position relative to the target line.
Full size table

Temporal filtering in tennis groundstrokes (shown in Table 2), is implemented as two-stage reduction of temporal Region of Interest (ROI) of a swing interval S j [startFrame, …, endFrame]. Temporal filtering is defined as composition of temporal and spatial operations of ROI to obtain stance angle: \( \alpha = f_{1}^{ROI} (S_{j} ) \circ f_{2}^{ROI} (St_{j} ) \circ f_{3}^{ROI} (Sttr_{j} ) \), where \( Sttr_{j} \subseteq St_{j} \subset S_{j} \,. \)

The video capture system was initialised so that x-axis is parallel to the target line.

The intermediate experimental results in Fig. 4 show stance computing as angle extraction from temporal region of interest within recognised motion patterns.

Fig. 4.
figure 4

Results of multi-stage temporal filtering of marker’s trajectories for hips and hand/wrist that holds a racquet. Characteristic discriminant motion patters for selected swing intervals of: (a) forehand and (b) backhand. Vertical lines indicate selected time period of playing hand marker displacement around the pelvis centre (virtual Body Centre) relative to target line.

Full size image

3.2 Angle Transformations Issues, Customisations and Visualisations

The need for angle transformation function transf(.) is to: (1) produce monotone angle transformation; and (2) convert from 3D to 2D and to positive oriented angles in transverse x–z plane (i.e. top 3D view) for viewing and algorithm validation that would be intuitive to a domain expert (as depicted in Fig. 5).

Fig. 5.
figure 5

Visualising right-handed 3D transverse x-z plane compared to x–y 2D system

Full size image

To accommodate intuitive angle expert’s validation for cases where ‘closed-stance’ forehand would change sign (in IVth quadrant) the transf(.) function would increase computed negative angle by 2 π (e.g. comparable to compass north bearing between 1° and 359°) (Fig. 6).

Fig. 6.
figure 6

Customisation of positive and negative stance quadrants (x, y) by increasing angle by 2π

Full size image

Similarly, the developers may choose an arbitrary angle of e.g. ideal square stance parallel to x-axis to be calculated and reported to user either as 0° or 90°.

Considering that the feet position angle within a swing event S j may not be restricted to the tangent function domain (-π/2, π/2) then tan(.) could result in producing zero-division at π/2 dividable angles, given that \( \tan (x) = \frac{\sin (x)}{\cos (x)} \) (Fig. 7).

Fig. 7.
figure 7

Positive monotone angle transformation example of arctan(.) function for selected angle \( \alpha \) change (a) from non-monotone (b) to monotone function (c).

Full size image

Customising transformation task to provide angle reporting alternative:

Enabling customisation alternatives to stance angle extraction supports mediating coaches’ disagreements e.g. associated with decision boundaries between square stance and closed stance of forehand. The transformation function (3), was customised so that the negative angles of forehand stance to be increased by π (instead of 2π as generally proposed). The outcome of customised transformation function (3), would result in angles α ≥ 0 for the open, semi-open and square stances, while α < 0 would indicate the closed stance. Nonetheless the closed stance single hand backhands would result in angles α > 0.

$$ \alpha = \left\{ {\begin{array}{*{20}c} {\left( {\frac{{\sum\limits_{i = newStartFrame}^{newEndFrame} {\arctan \left( {\frac{{\overrightarrow {Zssm} [i] - \overrightarrow {Zpsm} [i]}}{{\overrightarrow {Xpsm} [i] - \overrightarrow {Xssm} )[i]}}} \right)} }}{no\_of\_frames}} \right),if\{ BACKHAND\} } \\ {\left( {\frac{{\sum\limits_{i = newStartFrame}^{newEndFrame} {\arctan \left( {\frac{{\overrightarrow {Zpsm} [i] - \overrightarrow {Zssm} [i]}}{{\overrightarrow {Xssm} [i] - \overrightarrow {Xpsm} )[i]}}} \right)} }}{no\_of\_frames}} \right),if\{ FOREHAND\} \cap \{ \overrightarrow {(Xssm} [i] - \overrightarrow {Xpsm} )[i]) \ge 0\} } \\ {\left( {\frac{{\sum\limits_{i = newStartFrame}^{newEndFrame} {\arctan \left( {\frac{{\overrightarrow {Zpsm} [i] - \overrightarrow {Zssm} [i]}}{{\overrightarrow {Xssm} [i] - \overrightarrow {Xpsm} )[i]}}} \right)} }}{no\_of\_frames} + \pi } \right),if\{ FOREHAND\} \cap \{ \overrightarrow {(Xssm} [i] - \overrightarrow {Xpsm} )[i]) < 0\} } \\ \end{array} } \right. $$
(3)

4 Discussion, Limitation and Critique

Stance assessment may be assessed by a coach as holistic body position. To extend this study, hip rotation for example could reveal additional information related to stance an even to grip preferences (way holding a racquet to execute a particular swing type). In addition, shoulder turn pattern could improve safety, performance and help assessing e.g. serve return technique. When players are hitting the ball on the run, comparing the shoe tips position relative to the target line may not always be a reliable method in particular for low frame rate systems. Although two-hand backhand data were not captured, the applied concept of playing arm crossing the virtual centre of the body is applicable to both single- and two-handed backhands. To aid in customisation of stance classification and to address the issue raising from the running positions the future algorithm extensions may also take into account average hip angle and feet position before the intended impact with the ball.

5 Conclusions, Recommendations and Future Work

The experiments on presented analytical computational approaches show that it is possible from multi-camera video to: (1) quantify body stance orientation by extracting the angles around the swing impact zone and (2) recognise the groundstrokes from 3D motion data specific to tennis. For exergames and augmented coaching system development, the presented concepts, stance customisation and extraction experiments on tennis 3D data are generally applicable to diverse disciplines and activities nature including dancing, hitting, kicking, throwing, blocking, shooting and balancing. In general, body orientation towards arbitrary direction during a specific movement pattern may provide valuable information in predicting intended activity development. Body orientation monitoring and processing is also applicable to other cross disciplinary research, including surveillance and machine vision safety monitoring, e.g. pedestrian traffic collision avoidance. Future work will include advancing and extending application to rehabilitation devices to extract, monitor, assess and provide feedback in near-real time from captured kinematic chain sequence and other human ineligible parameters that may guide functions of augmented coaching systems. Future work on assessment of balance, kinetic/kinematic chain and movement fluidity would benefit from availability of big motion data sets, 3D motion acquisition systems and cross disciplinary surveillance research such as in the areas of sporting activities, human action and semantic interpretations of events [15–17].