A trajectory-based analysis of coordinated team activity in a basketball game

Abstract

This paper proposes a novel, trajectory-based approach to the automatic recognition of complex multi-player behavior in a basketball game. First, a probabilistic play model is applied to the player-trajectory data in order to segment the play into game phases (offense, defense, time out). In this way, both the temporal boundaries of the observed activity and its broader context are obtained. Next, the team’s activity is analyzed in more detail by detecting the key elements of basketball play. Following basketball theory, these key elements (starting formation, screen, and move) are the building blocks of basketball play, and therefore their temporal order is used to produce a semantic description of the observed activity. Finally, the activity is recognized by comparing its semantic description with the descriptions of manually defined templates, stored in a database. The effectiveness and robustness of the proposed approach is demonstrated on two championship games and 71 examples of three types of basketball offense.

Introduction

One of the main challenges in sports science is an objective analysis of player performance. While an individual player’s physical abilities can be readily tested in the laboratory, a team’s performance can only be observed during an actual game. This process may include advanced analysis methods, such as video recording and statistical analysis, but it nevertheless relies on observation and manual annotation by sports experts, with the potential risk of becoming too subjective. Additionally, manual annotation is a time-consuming and tedious task, mostly limited either to academic research or to the small number of teams that can afford a sufficient number of qualified experts. Moreover, some researchers have found [1] that even sports experts often cannot observe and recall all the details that can prove crucial for the correct interpretation of the results. This is the reason for an increasing volume of research concerned with the automatic or semi-automatic recognition and analysis of human behavior in sports. The ultimate goal of such research is to develop methods for the automatic interpretation and analysis of a team’s performance, which would present a concise summary of the team’s and the players’ strengths, weaknesses, and mistakes.

The main focus of this article is the challenge of observing a basketball game and interpreting the activity of the team on the court. The quality of the team has two important components. The first component encompasses the skills of the players, and is expressed as their technical knowledge. The second component is expressed as the overall team tactics. In order for a team to be successful, it needs individuals with excellent technical skills. Nevertheless, these individuals have to be able to act together as a group—a task that requires good coordination between the individual players and can only be achieved with a lot of training. Following this challenge, the focus of this article is on an analysis of coordinated activity in team sports, in particular basketball.

It is widely accepted [2] that the most successful players have the capability to react differently in similar situations. On the team level, the situation is similar—good teams can quickly change their tactics if needed. Such behavior prevents the opposing teams from preparing a good defense, and keeps the play interesting, as both teams have to continuously adapt to the situation on the court. This introduces a certain level of complexity and randomness to a team’s performance, and makes the design of an entirely automatic analysis system, which would recognize, understand, and grade every possible situation on the court, extremely difficult. However, due to the nature of a sport’s rules and rigorous player training, the motion of the players across the court is not entirely random, and it is reasonable to expect that it is possible to extract some common features of the team, especially when considering that coordinated team play is usually practised in advance.

The remainder of this paper is structured as follows. In the rest of this section, we give a short overview of the related work and present the concept of our approach. The methods for the segmentation and recognition of the complex multi-agent behavior are presented in Sections 2 and 3 the experimental results are presented. Finally, Section 4 draws the main conclusions and describes our future work.

A lot of work concerning trajectory-based motion modeling and analysis has been presented in recent years. For example, Johnson and Hogg [3], [4] presented two approaches for modeling variable, non-linear behavior. In their first approach a competitive learning, neural network was used on flow vectors from image sequences of pedestrians [3], and in the other, a probabilistic motion model was obtained using a Gaussian mixture model, representing the system state changes of pedestrian motion [4].

The more sophisticated, trajectory-based, multi-agent, action-recognition, and analysis approaches involve the compact representation and modeling of actions and interactions and their logical and temporal relations [5], [6], [7], [8], [9], [10]. Rao et al. [5] presented an approach to view invariant action-recognition that is capable of explaining an action in terms of meaningful action units called dynamic instants and intervals. Li and Woodham [6] presented a system for representing and reasoning about selected hockey plays based on trajectory data, augmented with domain-specific knowledge, such as forward/backward skating, puck possession, etc. Intille and Bobick [7] built models of football plays using belief networks and temporal graphs. A similar approach was used by Jug et al. [9] to assess team performance in basketball offense. The main contribution of the latter two approaches is the representation of multi-agent activity and recognition from noisy trajectory data. This is done by dividing the multi-agent activity into individual visually grounded, goal-based primitives that are probabilistically integrated with the low-order temporal and logical relationships. However, there are two main problems with such an approach. The first one is the need for a precise temporal segmentation of the analyzed trajectories. The second problem is the difficulty of building temporal and logical relationships, especially because of the many different parameters that need to be defined manually. Therefore, such an approach is not particularly suitable for cases when either a large quantity of data has to be analyzed or many different behavior models are used in the analysis.

Other sport-related research has focused on the content-based indexing of video footage [11], [12], learning motion models in golf [13], soccer [14], baseball [15], or choreography in dance [16], and acrobatics [17].

The long-term aim of the research described above is to provide athletes with the feedback they need to improve their kinematic skills. It can also be used to develop and monitor the correct execution of certain predefined mechanical actions (e.g. ball handling) or to learn the correct execution of more complex team activities.

In our work, we address the problem of a trajectory-based analysis of basketball, with the aim of overcoming the described problems. We chose our approach with expert sports knowledge in mind. Like with the procedure used in sports research, we perform a two-step analysis process, where the game is first segmented according to the phases of play (offense, defense, time out), and then each of the segments is analyzed in detail.

In the first step, a Gaussian mixture model is used to segment the continuous player trajectories into shorter game segments, corresponding to offense, defense, and time outs. This step provides us with both the roughly segmented boundaries and with the broader context of the game inside a particular segment. In the second step, we are able to recognize a particular type of basketball activity. In this article, we only focus on the recognition of the organized activity in the offensive part of the game. The basketball offense is the most interesting and widely studied [2], [18] part of the game for coaches and basketball experts. It is trained in advance and the set of learnt offenses for a particular team does not change significantly during the course of a single game.

To perform the recognition of activity, we developed three different trajectory-based detectors of the key basketball elements, which are the theoretical building blocks of any basketball play. These detectors are used to transform player trajectories, which represent the observed activity, into the sequence of symbols—the semantic description. The obtained symbols describe the actions and interactions between the players, and contain the notation of the observed element and its observed position on the court. To determine the similarity between the observed activity and the predefined activity template, we simply compute a Levenshtein distance between the sequence of symbols obtained from the trajectories of the players and the sequence of symbols obtained from the activity template. By repeating the template-trajectory matching procedure for every template available, we can find the template with the shortest distance to the sequence of symbols, obtained from the trajectory data. If the distance is below a certain threshold, we assign the label of the most similar template to the observed trajectory segment.

Besides removing the need for manual segmentation, an important contribution of our approach is the simplified process of providing expert knowledge in machine-suitable format. The method used by Intille and Bobick [7] requires the provision of expert knowledge in the form of temporal and logical relationships in a belief network for every activity that has to be recognized. This process is inappropriate for field experts (i.e., sports coaches), slow, and can be inaccurate or subjective because of the non-obvious relationships between the activity and the network structure [7], [9]. In our case, the specific structure of basketball activity is encoded in the form of activity templates, which can be represented graphically and are significantly better understood by an average sports expert.