Skip to main content
SpringerLink
Log in

Consensus-based trajectory estimation for ball detection in calibrated cameras systems

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

This paper considers the detection of the ball in team sport scenes observed with still or motion-compensated calibrated cameras. Foreground masks do provide primary cues to identify circular moving objects in the scene, but are shown to be too noisy to achieve reliable detections of weakly contrasted balls, especially when a single viewpoint is available, as often desired for reduced deployment cost. In those cases, trajectory analysis has been shown to provide valuable complementary information to differentiate true and false positives among the candidates detected by the foreground mask(s). In this paper, we focus on the detection of ball trajectory segments, exclusively from visual cues, without considering semantic reasoning about team play to connect those segments into long trajectories. We revisit several recent works, mainly the ones that we presented in Amit Kumar et al. (ICDSC, 1), Parisot and De Vleeschouwer (ICME, 2), and introduce a publicly available dataset to compare them. We conclude that randomized consensus-based methods are competitive compared to the alternative deterministic graph-based solutions, while offering the additional advantage to naturally extend to the cost-effective single-view scenario. As an original contribution, we also introduce a procedure to efficiently clean up the foreground mask in correlation-based methods like (Amit Kumar et al. in ICDSC, 1) and a nonlinear rank-order filter to merge the foreground cues from multiple viewpoints. We also derive recommendations regarding the camera positioning and the buffering needs of a real-time acquisition system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. In this paper, the player positions are computed as in [4].

  2. The use of a small threshold induces a significant amount of false positive foreground pixels in the mask, which makes the cleanup stage, as well as the subsequent template-based correlation and ballistic trajectory analysis, especially helpful to avoid false ball detections.

  3. Each player is approximated by a 40 cm (width) \(\times\) 180 cm (height) rectangular box.

  4. To prevent the unfounded removal of ball candidates, the operating point of the player detector is chosen to maintain a very small false positive rate, at the cost of a relatively moderate detection rate.

  5. Assuming that the single viewpoint is at least 15 m away from the field, which is common in practice, the actual template size lies within a margin of less than \(30\,\%\) around the constant-size template, chosen to fit the ball silhouette when the ball is in the field center.

  6. False detections generally appear in bursts, and candidate location inaccuracies result from calibration and synchronization errors in multi-view setups.

  7. The dataset is unique in terms of relevance because (1) the camera placement has been selected for production purpose rather than to detect the ball, and (2) the lightning is far from optimal [26, 40], but representative of what is commonly encountered in practice.

  8. For some M, the order r might be larger than the number of views in which M is visible. In that case, the order of the filter is set to the number of available views.

  9. Note that the first rank-order filter has not been depicted because it leads to poor performance, due to the noisy nature of individual foreground masks, but also due to the fact that a single-view 2D location is associated with many 3D positions.

  10. Detecting the ball in those situations is important regarding sport analysis because it complements the cues derived from players displacements (see [26] and [44]).

  11. We neglect the complexity associated with the RANSAC-based ball trajectory estimation, since it runs over a relatively small number of preselected ball position candidates (\({\le} T\) for the Corr approach and \({\le} 3T\) for the CCA approach, T being the length of the window, typically \(T=24\)).

  12. \(14 \times 10^6\) = 12,000 fps \(\times\) (40 \(\times\) 30 \(\times\) 30 )/30 fps.

References

  1. Amit Kumar, K.C., Parisot, P., De Vleeschouwer, C.: Demo: Spatio-temporal template matching for ball detection. In: ICDSC (2011)

  2. Parisot, P., De Vleeschouwer, C.: Graph-based filtering of ballistic trajectory. In: ICME (2011)

  3. Parisot, P., Sevilmiş, B., De Vleeschouwer C.: Training with corrupted labels to reinforce a probably correct teamsport player detector. In: ACIVS (2013)

  4. Delannay, D., Danhier, N., De Vleeschouwer, C.: Detection and recognition of sports (wo)men from multiple views. In: ACM/IEEE ICDSC, pp. 1–7 (2009)

  5. Fernandez, I., Chen, F., Lavigne, F., Desurmont, X., De Vleeschouwer, C.: Browsing sport content through an interactive H.264 streaming session. In: MMEDIA, pp. 155–161 (2010)

  6. Chen, F., Delannay, D., De Vleeschouwer, C.: An autonomous framework to produce and distribute personalized team-sport video summaries: a basket-ball case study. IEEE Trans. Multimed. 13(6), 1381–1394 (2011)

    Article  Google Scholar 

  7. Chen, F., De Vleeschouwer, C.: Personalized production of basketball videos from multi-sensored data under limited display resolution. CVIU 114(6), 667–680 (2010)

    Google Scholar 

  8. Keemotion: Keemotion production technology. http://www.keemotion.com

  9. Sun, L., De Neyer, Q., De Vleeschouwer, C.: Multimode spatiotemporal background modeling for complex scenes. In: EUSIPCO, pp. 165–169 (2012)

  10. Stauffer, C., Grimson, W.: Adaptive background mixture models for real-time tracking. In: CVPR, pp. 246–252 (1999)

  11. Horprasert, T., Harwood, D., Davis, L.: A statistical approach for real-time robust background subtraction and shadow detection. In: ICCV, pp. 1–19 (1999)

  12. Reddy, V., Sanderson, C., Lovell, B.: Improved foreground detection via block-based classifier cascade with probabilistic decision integration. IEEE TCSVT 23(1), 83–93 (2013)

    Google Scholar 

  13. Zhou, J., Hoang, J.: Real time robust human detection and tracking system. In: CVPR (2005)

  14. Cavallaro, A., Steiger, O., Ebrahimi, T.: Semantic video analysis for adaptive content delivery and automatic description. IEEE TCSVT 15(10), 1200–1209 (2005)

    Google Scholar 

  15. Khan, S., Shah, M.: A multiview approach to tracking people in crowded scenes using a planar homography constraint. In: ECCV, vol. 4, pp. 133–146 (2006)

    Chapter  Google Scholar 

  16. Fleuret, F., Berclaz, J., Lengagne, R., Fua, P.: Multi-camera people tracking with a probabilistic occupancy map. IEEE Trans. PAMI 30(2), 267–282 (2008)

    Article  Google Scholar 

  17. Alahi, A., Jacques, L., Boursier, Y., Vandergheynst, P.: Sparsity driven people localization with a heterogeneous network of cameras. J. MIV 41(1–2), 39–58 (2011)

    MathSciNet  MATH  Google Scholar 

  18. Dollar, P., Wojek, C., Schiele, B., Perona, P.: Pedestrian detection: a benchmark. In: CVPR (2009)

  19. Felzenszwalb, P., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. PAMI 32(9), 1627–1645 (2010)

    Article  Google Scholar 

  20. Viola, P., Jones, M.: Robust real-time object detection. In: International Workshop on SCTV (2001)

  21. Hawk-Eye: Hawk-Eye ball tracking system. http://www.hawkeyeinnovations.co.uk/page/sports-officiating

  22. Yan, F., Christmas, W., Kittler, J.: A tennis ball tracking algorithm for automatic annotation of tennis match. In: BMVC, pp. 619–628 (2005)

  23. Choi, K., Seo, Y.: Probabilistic tracking of the soccer ball. In: ECCV, pp. 50–60 (2004)

  24. Public, Apidis dataset, ball detection ground-truth, and code samples. http://sites.uclouvain.be/ispgroup/index.php/Softwares/APIDIS

  25. Zhang, Y., Lu, H., Xu, C.: Collaborate ball and player trajectory extraction in broadcast soccer video. In: International Conference on Pattern Recognition (2008)

  26. Wang, X., Ablavsky, V., Ben Shitrit, H., Fua, P.: Take your eyes off the ball: improving ball-tracking by focusing on team play. CVIU 119, 102–115 (2014)

    Google Scholar 

  27. Wang, X., Tretken, E., Fleuret, F., Fua, P.: Tracking interacting objects optimally using integer programming. In: ECCV (2014)

  28. Lampert, C., Peters, J.: Real-time detection of colored objects in multiple camera streams with off-the-shelf hardware components. J. Real Time Image Proc. 7(1), 31–41 (2012)

    Article  Google Scholar 

  29. Ren, J., Orwell, J., Jones, G., Xu, M.: Real-time modeling of 3D soccer ball trajectories from multiple fixed cameras. IEEE Trans. Circuits Syst. Video Technol. 18, 350–362 (2008)

    Article  Google Scholar 

  30. Labayen, M., Olaizola, I., Aginako, N., Florez, J.: Accurate ball trajectory tracking and 3D visualization for computer-assisted sports broadcast. Multimed. Tools Appl. 73(3), 1819–1842 (2014)

    Article  Google Scholar 

  31. Chen, H.-T., Tien, M.-C., Chen, Y.-W., Tsai, W.-J., Lee, S.-Y.: Physics-based ball tracking and 3D trajectory reconstruction with applications to shooting location estimation in basketball video. J. VCIR 20(3), 204–216 (2009)

    Google Scholar 

  32. Yan, F., Kostin, A., Christmas, W., Kittler, J.: A novel data association algorithm for object tracking in clutter with application to tennis video analysis. In: CVPR, pp. 634–641 (2006)

  33. Zhou, X., Huang, Q., Xie, L., Cox, S.J.: A two layered data association approach for ball tracking. In: ICAPSS, p. 5 (2013)

  34. Chen, H.-T., Tsai, W.-J., Lee, S.-Y., Yu, J.-Y.: Ball tracking and 3D trajectory approximation with applications to tactics analysis from single-camera volleyball sequences. Multimed. Tools Appl. 60(3), 641–667 (2012)

    Article  Google Scholar 

  35. Chakraborty, B., Meher, S.: A real-time trajectory-based ball detection-and-tracking framework for basketball video. J. Opt. 42(2), 156–170 (2013)

    Article  Google Scholar 

  36. Poiesi, F., Daniyal, F., Cavallaro, A.: Detector-less ball localization using context and motion flow analysis. In: ICIP (2010)

  37. Kim, K., Grundmann, M., Shamir, A., Matthews, I., Hodgins, J., Essa, I.: Motion fields to predict play evolution in dynamic sport scenes. In: IEEE Conference on Computer Vision Pattern Recognition (2010)

  38. Ohno, Y., Miura, J., Shirai, Y.: Tracking players and estimation of the 3D position of a ball in soccer game. In: International Conference on Pattern Recognition (2000)

  39. Leo, M., Mosca, N., Spagnolo, P., Mazzeo, P., D’Orazio, T., Distant, A.: Real-time multi-view analysis of soccer matches for understanding interactions between ball and players. In: ACM International Conference on Image and Video Retrieval (2008)

  40. Maksai, A., Wang, X., Fua, P.: What players do with the ball: a physically constrained interaction modeling. In: CVPR (2016)

  41. Viola, P., Jones, M.: Robust real-time object detection. IJCV 57(2), 137–154 (2004)

    Article  Google Scholar 

  42. SportVU: STATS SportVU technology. http://www.stats.com/sportvu/sportvu.asp

  43. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  44. Gross, J.T., Yellen, J.: Graph Theory and its Applications. CRC Press, Boca Raton (1999)

    MATH  Google Scholar 

  45. Videos, Ball tracks. http://sites.uclouvain.be/ispgroup/index.php/Research/BallTrac

  46. Chen, F., Delannay, D., De Vleeschouwer, C.: Multi-sensored vision for autonomous production of personalized video summaries. In: International ICST Conference on User Centric Media (2010)

  47. Perše, M., Kristan, M., Kovačič, S., Vučkovič, G., Perš, J.: A trajectory-based analysis of coordinated team activity in a basketball game. CVIU 113, 612–621 (2009)

    Google Scholar 

  48. Porikli, F.: Integral histogram: a fast way to extract histograms in Cartesian spaces. In: IEEE Conference on Computer Vision Pattern Recognition (2005)

Download references

Acknowledgments

Part of this work has been funded by the Belgian NSF and the Walloon region projects SPORTIC and PTZ-PILOT.

Author information

Authors and Affiliations

  1. ICTEAM-ELEN, Université catholique de Louvain, Place du Levant 2, 1348, Louvain-La-Neuve, Belgium

    Pascaline Parisot & Christophe De Vleeschouwer

Authors
  1. Pascaline Parisot
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christophe De Vleeschouwer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christophe De Vleeschouwer.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Parisot, P., De Vleeschouwer, C. Consensus-based trajectory estimation for ball detection in calibrated cameras systems. J Real-Time Image Proc 16, 1335–1350 (2019). https://doi.org/10.1007/s11554-016-0638-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-016-0638-3

Keywords

Search

Navigation