Skip to main content
Springer Nature Link
Log in

Modeling spatiotemporal relationships between moving objects for event tactics analysis in tennis videos

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

Evolution of spatial relationships between objects often provides important clues for semantic video analysis. We present a symbolic representation that describes spatiotemporal characteristics and facilitates tactics detection based on string matching. To find typical spatiotemporal patterns of a targeted tactic, we organize training sequences as a tree, and effectively discover frequent patterns from the structure. Tactics detection is conducted by comparing a given test sequence with these frequent patterns. To realize the proposed idea, we develop elaborate audio/video processes to transform broadcasting tennis videos into symbolic sequences, and comprehensively tackle event detection and tactics analysis. We experiment on ten most important tennis championships in the year 2008, and report promising detection results on seven events/tactics. We demonstrate not only the effectiveness of the proposed methods, but also study the impacts brought by the results of tactics analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Cai R, Lu L, Zhang H-J, Cai LH (2003) Highlight sound effects detection in audio stream. In Proceedings of IEEE International Conference on Multimedia & Expo, vol. 3, 37–40

  2. Chang C-C, Lin C-J (2001) LIBSVM: a library for support vector machine. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm

  3. Chang S-F, Chen W, Meng HJ, Sundaram H, Zong D (1998) A fully automated content-based video search engine supporting spatiotemporal queries. IEEE Trans Circuits Syst Video Technol 8(5):602–615

    Article  Google Scholar 

  4. Cheng W-H, Chu W-T, Wu J-L (2003) Semantic context detection based on hierarchical audio models. In Proceedings of the ACM SIGMM International Workshop on Multimedia Information Retrieval, pp 109–115

  5. Choi J, Joen WJ, Lee S-C (2008) Spatio-temporal pyramid matching for sports videos. In Proceedings of ACM International Conference on Multimedia Information Retrieval, pp. 291–297

  6. Chu W-T, Wu J-L (2008) Explicit semantic events detection and development of realistic applications for broadcasting baseball videos. Multimedia Tools and Applications 38(1):27–50

    Article  MathSciNet  Google Scholar 

  7. Chu W-T, Tien M-C, Wang Y-T, Chou C-W, Hsieh K-Y, Wu J-L (2007) Event detection in tennis matches based on real-world audiovisual cues. In Proceedings of the 20th Computer Vision, Graphics, and Image Processing Conference, pp 541-548

  8. Duan L-Y, Xu M, Chua T-S, Tian Q, Xu C-S (2003) A mid-level representation framework for semantic video analysis. In Proceedings of ACM Multimedia, pp. 33–44

  9. Duan L-Y, Xu M, Tian Q, Xu C-S, Jin JS (2005) A unified framework for semantic shot classification in sports video. IEEE Trans Multimedia 7(6):1066–1083

    Article  Google Scholar 

  10. Ekin A, Tekalp AM, Mehrota R (2003) Automatic soccer video analysis and summarization. IEEE Trans Image Process 12(7):796–807

    Article  Google Scholar 

  11. Farin D, Krabbe S, de With PHN, Effelsberg W (2004) Robust camera calibration for sport videos using court models. In Proceedings of SPIE Storage and Retrieval Methods and Applications for Multimedia, vol. 5307, 80–91

  12. Galmar E, Huet B (2008) Spatiotemporal modeling and matching of video shots. In Proceedings of 1st ICIP workshop on Multimedia Information Retrieval: New Trends and Challenges

  13. Han M, Hua W, Xu W, Gong Y (2002) An integrated baseball digest system using maximum entropy method. In Proceedings of ACM Multimedia, pp 347–350

  14. Han J, Farin D, de With PHN (2006) Multi-level analysis of sports video sequences. In Proceedings of SPIE Conference on Multimedia Content Analysis, Management, and Retrieval

  15. Hanjalic A (2002) Shot-boundary detection: unraveled and resolved? IEEE Trans Circuits Syst Video Technol 12(2):90–105

    Article  Google Scholar 

  16. Hartley R, Zisserman A (2000) Multiple view geometry in computer vision. Cambridge University Press

  17. Huang Y-P, Chiou C-L, Sandnes FE (2009) An intelligent strategy for the automatic detection of highlights in tennis video recordings. Expert Syst Appl 36:9907–9918

    Article  Google Scholar 

  18. Kijak E, Gravier G, Oisel L, Gros P (2006) Audiovisual integration for tennis broadcast structuring. Multimedia Tools and Applications 30:289–311

    Article  Google Scholar 

  19. Kolonias I, Christmas W, Kittler J (2004) Automatic evolution tracking for tennis matches using an HMM-based architecture. In Proceedings of IEEE Workshop on Machine Learning for Signal Processing, pp 615–624

  20. Lan D-J, Ma Y-F, Ma W-Y, Zhang H-J (2004) Spatio-temporal pattern mining in sports video. In Proceedings of Pacific-Rim Conference on Multimedia, pp 306–313

  21. Leonardi R, Migliorati P, Prandini M (2004) Semantic indexing of soccer audio-visual sequences: a multimodal approach based on controlled Markov chains. IEEE Trans Circuits Syst Video Technol 14(5):634–643

    Article  Google Scholar 

  22. Liu Y, Jiang S, Ye Q, Gao W, Huang Q (2005) Playfield detection using adaptive GMM and its applications. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 2, 421–424

  23. Pallavi V, Mukherjee J, Majumdar AK, Sural S (2008) Graph-based multiplayer detection and tracking in broadcast soccer videos. IEEE Trans Multimedia 10(5):794–805

    Article  Google Scholar 

  24. Pantic M, Patras I, Valstar MF (2005) Learning spatio-temporal models of facial expressions. In Proceedings of International Conference on Measuring Behaviour

  25. Rea N, Dahyot R, Kokaram A (2005) Classification and representation of semantic content in broadcast tennis videos. In Proceedings of IEEE International Conference on Image Processing, vol. 3, pp. 1204–1207

  26. Rui Y, Gupta A, Acero A (2000) Automatically extracting highlights for TV baseball programs. In Proceedings of ACM Multimedia, pp 105–115

  27. Sadlier DA, O’connor NE (2005) Event detection in field sports videos using audio-visual features and a support vector machine. IEEE Trans Circuits Syst Video Technol 15(10):1225–1233

    Article  Google Scholar 

  28. Snoek CGM, Worring M, Geusebroek J-M, Koelma DC, Seinstra FJ, Smeulders AWM (2006) The semantic pathfinder: using an authoring metaphor for generic multimedia indexing. IEEE Trans Pattern Anal Mach Intell 28(10):1678–1689

    Article  Google Scholar 

  29. Song X, Fan G (2007) Selecting salient frames for spatiotemporal video modeling and segmentation. IEEE Trans Image Process 16(2):3035–3046

    Article  MathSciNet  Google Scholar 

  30. Sudhir G, Lee JC, Jain AK (1998) Automatic classification of tennis videos for high-level content-based retrieval. In Proceedings of International Workshop on Content-Based Access of Image and Video Databases, pp. 81–90

  31. Tagagi S, Hattori S, Yokoyama K, Kodate A, Tominaga H (2003) Sports video categorizing method using camera motion parameters. In Proceedings of IEEE International Conference on Multimedia & Expo, vol. 2, pp 461–464

  32. United States Tennis Association (1996) Tennis tactics—winning patterns of play. Human Kinetics Publishers

  33. Wang JR, Parameswaran N (2005) Analyzing tennis tactics from broadcasting tennis video clips. In Proceedings of International Multimedia Modelling Conference, pp 102–106

  34. Wang Y, Liu Z, Huang JC (2000) Multimedia content analysis using both audio and visual cues. IEEE Signal Process Mag 17(6):12–36

    Article  Google Scholar 

  35. Wang P, Cui R, Yang S-Q (2004) A tennis video indexing approach through pattern discovery in interactive process. In Proceedings of Pacific-Rim Conference on Multimedia, pp 49–56

  36. Xie L, Xu P, Chang S-F, Divakaran A, Sun H (2004) Structure analysis of soccer video with domain knowledge and hidden Markov models. Pattern Recogn Lett 26(7):767–775

    Article  Google Scholar 

  37. Xu H, Chua T-S (2006) Fusion of AV features and external information sources for event detection in team sports video. ACM Transactions on Multimedia Computing, Communications, and Applications 2(1):44–67

    Article  Google Scholar 

  38. Xu M, Duan L-Y, Xu C-S, Tian Q (2003) A fusion scheme of visual and auditory modalities for event detection in sports video. In Proceedings IEEE International Conference on Acoustics, Speech, & Signal Processing, vol. 3, pp. 189–192

  39. Yu X, Xu C, Leong HW, Tian Q, Tang Q, Wan KW (2003) Trajectory-based ball detection and tracking with applications to semantic analysis of broadcast soccer video. In Proceedings of ACM Multimedia, pp 11–20

  40. Zhang D, Chang S-F (2002) Event detection in baseball video using superimposed caption recognition. In Proceedings of ACM Multimedia, pp. 315–318

  41. Zhu G, Xu C, Huang Q, Gao W (2006) Automatic multi-player detection and tracking in broadcast sports video using support vector machine and particle filter. In Proceedings of IEEE International Conference on Multimedia & Expo, pp 1629–1632

  42. Zhu, G., Huang, Q., Xu, C., Rui, Y., Jiang, S., Gao, W., and Yao, H. (2007) Trajectory based event tactics analysis in broadcast sports video. In Proceedings of ACM Multimedia, pp. 58–67

  43. Zhu G, Huang Q, Xu C, Xing L, Gao W, Yao H (2007) Human behavior analysis for highlight ranking in broadcast racket sports video. IEEE Trans Multimedia 9(6):1167–1182

    Article  Google Scholar 

  44. Zhu G, Xu C, Huang Q, Liu H (2008) Event tactic analysis based on player and ball trajectory in broadcast video. In Proceedings of ACM International Conference on Image and Video Retrieval, pp 515–523

Download references

Acknowledgements

This work was partially supported by the National Science Council of ROC under NSC 96-2218-E-194-005 and 97-2221-E-194-050. The authors would like to thank anonymous reviewers for giving valuable comments, and thank Ming-Chun Tien, Yi-Tang Wang, Chen-Wei Chou, Kuei-Yi Hsieh, and Ja-Ling Wu for co-developing former version of the system.

Author information

Authors and Affiliations

  1. Department of Computer Science and Information Engineering, National Chung Cheng University, Min-Hsiung, Chia-Yi, Taiwan

    Wei-Ta Chu & Wen-Ho Tsai

Authors
  1. Wei-Ta Chu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wen-Ho Tsai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei-Ta Chu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chu, WT., Tsai, WH. Modeling spatiotemporal relationships between moving objects for event tactics analysis in tennis videos. Multimed Tools Appl 50, 149–171 (2010). https://doi.org/10.1007/s11042-009-0363-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-009-0363-z

Keywords