Skip to main content
SpringerLink
Log in

Bayesian belief network based broadcast sports video indexing

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper presents a probabilistic Bayesian belief network (BBN) method for automatic indexing of excitement clips of sports video sequences. The excitement clips from sports video sequences are extracted using audio features. The excitement clips are comprised of multiple subclips corresponding to the events such as replay, field-view, close-ups of players, close-ups of referees/umpires, spectators, players’ gathering. The events are detected and classified using a hierarchical classification scheme. The BBN based on observed events is used to assign semantic concept-labels to the excitement clips, such as goals, saves, and card in soccer video, wicket and hit in cricket video sequences. The BBN based indexing results are compared with our previously proposed event-association based approach and found BBN is better than the event-association based approach. The proposed scheme provides a generalizable method for linking low-level video features with high-level semantic concepts. The generic nature of the proposed approach in the sports domain is validated by demonstrating successful indexing of soccer and cricket video excitement clips. The proposed scheme offers a general approach to the automatic tagging of large scale multimedia content with rich semantics. The collection of labeled excitement clips provide a video summary for highlight browsing, video skimming, indexing and retrieval.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Agarwal R, Shrikant R (1994) Fast algorithm for mining association rules. In: Int conf on very large data bases, pp 487–499

  2. Aigrain P, Zhang H, Petkovic D (1996) Representation and retrieval of visual media: a state-of-the-art review. Int J Multimed Tools Appl 3:179–202

    Google Scholar 

  3. Ancona N, Cicirelli G, Branca A, Distante A (2001) Goal detection in football by using support vector machines for classification. Int. Joint Conf Neural Netw 1:611–616

    Google Scholar 

  4. Assfalg J, Bertini M, Colombo C, Bimbo A, Nunziati W (2003) Semantic annotation of soccer videos: automatic highlights identification. J Comput Vis Image Und 92(2–3):285–305

    Article  Google Scholar 

  5. Baillie M, Jose JM (2003) Audio-based event detection for sports video. Lect Notes Comput Sci 2728:61–66

    Article  Google Scholar 

  6. Babaguchi N, Kawai Y, Ogura T, Kitahashi T (2004) Personalized abstraction of broadcasted american football video by highlight selection. IEEE Trans Multimedia 6(4):107–109

    Article  Google Scholar 

  7. Barnard M, Odobez J (2003) Multi-modal audio-visual event recognition for football analysis. In: IEEE workshop neural networks for signal processing, pp 469–478

  8. Bertini M, Cucchiara R, Bimbo AD, Prati A (2005) An integrated framework for semantic annotation and adaptation. Int J Multimed Tools Appl 26:345–363

    Article  Google Scholar 

  9. Christel M, Stevens S, Kanade T, Mauldin M, Reddy R, Wactlar H (1995) Techniques for the creation and exploration of digital video libraries. Int J Multimed Tools Appl 2:501–533

    Google Scholar 

  10. Cheng C, Hsu C (2006) Fusion of audio and motion information on HMM based highlight extraction for baseball games. IEEE Trans Multimedia 8(3):585–599

    Google Scholar 

  11. Dimitrova N, Zhang H, Shahraray B, Sezan I, Huang T, Zakhor A (2002) Applications of video-content analysis and retrieval. IEEE Multimed 9(3):42–55

    Article  Google Scholar 

  12. Ding Y, Fan G (2007) Segmental hidden markov model for view-based sports video analysis. In: IEEE inf. conf. on computer vision and pattern recognition

  13. Duan L, Xu M, Chua T, Tian Q, Xu C (2003) A mid-level repre sentation framework for semantic sports video analysis. In: ACM int. conf on multimedia

  14. Duan L, Xu M, Tian Q, Xu C, Jin J (2005) A unified framework for semantic shot classification in sports video. IEEE Trans Multimedia 7(6):1066–1083

    Article  Google Scholar 

  15. Ekin A, Tekalp AM, Mehrotra R (2003) Automatic soccer video analysis and summarization. IEEE Trans Image Process 12(7):796–807

    Article  Google Scholar 

  16. Hanjalic A (2003) Generic approach to highlight extraction from a sport video. IEEE Int Conf Image Process 1:1–4

    Google Scholar 

  17. Hauptmann AG, Smith M (1995) Text, speech and vision for video segmentation: the informedia project. Writing notes of IJCAI workshop on intelligent multimedia information retrieval, pp 17–22

  18. Huang CL, Shih HC, Chao CY (2006) Semantic analysis of soccer video using dynamic Bayesian network. IEEE Trans Multimedia 8(4):749–760

    Article  Google Scholar 

  19. Jaakkola TS, Jordan MI (2000) Bayesian parameter estimation via variational methods. Stat Comput 10:25–37

    Article  Google Scholar 

  20. Jordan MI (2004) Graphical models. Stat Sci 19:140–155 (Special issue on Bayesian statistics)

    Article  MATH  Google Scholar 

  21. Jung C, Kim J (2009) Player information extraction for semantic annotation in golf videos. IEEE Trans Broadcast 55(1):79–83

    Article  Google Scholar 

  22. Kokaram A, Rea N, Dahyot R, Tekalp M, Bouthemy P, Gros P, Sezan I (2006) Browsing sports video: trends in sports-related indexing and retrieval work. IEEE Signal Process Mag 23(2):47–58

    Article  Google Scholar 

  23. Kolekar MH, Sengupta S (2006) Event-importance based customized and automatic cricket highlight generation. In: IEEE int conf multimedia expo, pp 1617–1620

  24. Kolekar MH, Sengupta S (2006) A hierarchical framework for generic sports video classification, Lecture notes on computer science, vol 3852. Springer, Heidelberg, pp 633–642

    Google Scholar 

  25. Kolekar MH, Sengupta S (2010) Semantic concept mining in cricket videos for automated highlight generation. Multimed Tools Appl 47(3):545–579

    Article  Google Scholar 

  26. Kolekar MH, Palaniappan K, Sengupta S (2008) Semantic event detection and classification in cricket video sequences. In: IEEE indian conf. computer vision, graphics and image processing, pp 382–389

  27. Kolekar MH, Palaniappan K, Sengupta S, Seetharaman G (2009) Semantic concept mining based on hierarchical event detection for soccer video indexing. Int J Multimed 4(5):298–312

    Google Scholar 

  28. Kopparapu SK, Desai UB (2001) Bayesian approach to image interpretation, vol 616. Kluwer Academic Publisher

  29. Lefevre S, Maillard B, Vincent N (2002) Three classes segmentation for analysis of football audio sequences. Int Conf Digital Signal Process 2:975–978

    Google Scholar 

  30. Leonardi R, Migliorati P, Prandini M (2004) Semantic indexing of soccer audio-visual sequences: a multimodal approach based on controlled Markov chains. IEEE Trans Circuits Syst Video Technol 14(5):634–643

    Article  Google Scholar 

  31. Li B, Pan H, Sezan I (2003) A general framework for sports video summarization with its application to soccer. IEEE Int Conf Acoust Speech Signal Process 3:169–172

    Google Scholar 

  32. Li Y, Narayanan S, Kuo CCJ (2004) Content-based movie analysis and indexing based on audiovisual cues. IEEE Trans Circuits Syst Video Technol 14(8):1073–1085

    Article  Google Scholar 

  33. Li Y, Dore A, Orwell J (2005) Evaluating the performance of systems for tracking football players and ball. In: IEEE int. conf. advanced video and signal based surveillance

  34. Mei T, Ma YF, Zhou HQ, Ma WY, Zhang HJ (2005) Sports video mining with mosaic. In: IEEE—multimedia modeling conference, pp 107–114

  35. Narayana M, Haverkamp D (2007) A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video. In: IEEE inf. conf. on computer vision and pattern recognition, pp 1–8

  36. Nillius P, Sullivan J, Carlsson S (2006) Multi-target tracking-linking identities using Bayesian network inference. In: IEEE inf. conf. on computer vision and pattern recognition, vol 2

  37. Rui Y, Gupta A, Acero A (2000) Automatically extracting highlights for TV baseball programs. ACM Multimedia 105–115

  38. Sadlier D, O’Connor N (2005) Event detection in field sports video using audio-visual features and a support vector machine. IEEE Trans Circuits Syst Video Technol 15(10):1225–1233

    Article  Google Scholar 

  39. Sankar KP, Pandey S, Jawahar CV (2006) Text driven temporal segmentation of cricket videos. Int Conf Pattern Recognit 4338:433–444

    Google Scholar 

  40. Shih HC, Huang CL (2005) MSN: statistical understanding of broadcasted sports video using multilevel semantic network. IEEE Trans Broadcast 51(4):449–459

    Article  Google Scholar 

  41. Sudhir G, Lee JCM, Jain AK (1998) Automatic classification of tennis video for high-level content-based retrieval. In: IEEE int. workshop content-based access of image and video databases, pp 81–90

  42. Sun X, Jin G, Huang M, Xu G (2003) Bayesian network based soccer video event detection and retrieval. In: Multispectral image processing and pattern recognition

  43. Tsin Y, Collins RT, Ramesh V, Kanade T (2001) Bayesian color constancy for outdoor object recognition. In: IEEE inf. conf. on computer vision and pattern recognition, pp 1132–1139

  44. Wan K, Xu C (2004) Recent soccer highlight generation with a novel dominant speech feature extractor. IEEE Int Conf Multimed Expo 1:591–594

    Google Scholar 

  45. Wang P, Cai R, Yang S (2004) A tennis video indexing approach through pattern discovery in interactive process. Lect Notes Comput Sci 3331:49–56

    Article  Google Scholar 

  46. Wang J, Chng E, Xu C, Lu H, Tian Q (2007) Generation of personalized music sports video using multimodal cues. IEEE Trans Multimedia 9(3):576–588

    Article  Google Scholar 

  47. Xie L, Chang SF, Divakaran A, Sun H (2002) Structure analysis of soccer video with hidden Markov models. IEEE Int Conf Acoust Speech Signal Process 4:4096–4099

    Google Scholar 

  48. Xu H, Chau T (2004) The fusion of audio-visual features and external knowledge for event detection in team sports video. In: ACM SIGMM int. multimedia workshop on multimedia information retrieval, pp 127–134

  49. Xu M, Orwell J, Jones G (2004) Tracking football players with multiple cameras. IEEE Int Conf Image Process 5:2909–2912

    Google Scholar 

  50. Xu C, Wang J, Lu H, Zhang Y (2008) A novel framework for semantic annotation and personalized retrieval of sports video. IEEE Trans Multimedia 10(3):421–436

    Article  Google Scholar 

  51. Xiong Z, Radhakrishnan R, Divakaran A, Huang TS (2003) Audio events detection based highlights extraction from baseball, golf, soccer games in a unified framework. IEEE Int Conf Acoust Speech Signal Process 5:632–635

    Google Scholar 

  52. Yu T, Zhang Y (2001) Retrieval of video clips using global motion information. Electron Lett 37(14):893–895

    Article  Google Scholar 

  53. Zhu X, Wu X, Elmagarmid AK, Feng Z, Wu L (2005) Video data mining: semantic indexing and event detection from the association perspective. IEEE Trans Knowl Data Eng 17(5):665–677

    Article  Google Scholar 

  54. Zhu G, Huang Q, Xu C, Xing L, Gao W, Yao H (2007) Human behavior analysis for highlight ranking in broadcast racket sports video. IEEE Trans. Multimedia 9(6):1167–1182

    Article  Google Scholar 

Download references

Acknowledgement

Author wish to thank Prof Somnath Sengupta and Prof. K. Palaniappan for their guidance and valuable suggestions.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Indian Institute of Technology, Patna, India

    Maheshkumar H. Kolekar

Authors
  1. Maheshkumar H. Kolekar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maheshkumar H. Kolekar.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kolekar, M.H. Bayesian belief network based broadcast sports video indexing. Multimed Tools Appl 54, 27–54 (2011). https://doi.org/10.1007/s11042-010-0544-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-010-0544-9

Keywords

Search

Navigation