Skip to main content
SpringerLink
Log in

Automatic composition of broadcast sports video

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

This study examines an automatic broadcast soccer video composition system. The research is important as the ability to automatically compose broadcast sports video will not only improve broadcast video generation efficiency, but also provides the possibility to customize sports video broadcasting. We present a novel approach to the two major issues required in the system’s implementation, specifically the camera view selection/switching module and the automatic replay generation module. In our implementation, we use multi-modal framework to perform video content analysis, event and event boundary detection from the raw unedited main/sub-camera captures. This framework explores the possible cues using mid-level representations to bridge the gap between low-level features and high-level semantics. The video content analysis results are utilized for camera view selection/switching in the generated video composition, and the event detection results and mid-level representations are used to generate replays which are automatically inserted into the broadcast soccer video. Our experimental results are promising and found to be comparable to those generated by broadcast professionals.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Leonardi, R., Migliorati, P., Prandini, M.: Semantic indexing of soccer audio-visual sequences: a multimodal approach based on controlled markov chains. IEEE Trans. Circ. Syst. Video Technol. (CSVT) 14(5), 34–43 (2004)

    Google Scholar 

  2. Xie, L., Xu, P., Chang, S.-F., Divakaran, A., Sun, H.: Structure analysis of soccer video with domain knowledge and hidden markov models. Pattern Recogn. Lett. 24 (2003)

  3. Adami, N., Leonardi, R., Migliorati, P.: An overview of multi-modal techniques for the characterization of sport programmes. In: Proceedings of SPIE-VCIP’03, pp. 1296–1306 (2003)

  4. Hauptmann, A., Smith, M.: Text, speech, and vision for video segmentation: the informedia project. In: AAAI Symposium on Computational Models for Integrating Language and Vision, pp. 10–12 (1995)

  5. Casares, J.P., Long, A.C., Myers, B.A., Bhatnagar, R., Stevens, S.M., Dabbish, L., Yocum, D., Corbett, A.T.: Simplifying video editng using metadata. In: Symposium on Designing Interactive Systems 2002, pp. 157–166 (2002)

  6. Davis, M.: Editing out video editing. IEEE Trans. MultiMedia 10(2), 54–64 (2003)

    Article  Google Scholar 

  7. Wan, K., Lim, J., Xu, C., Yu, X.: Real-time camera field-view tracking in soccer video, Vol. 3. In: Proceedings of IEEE ICASSP’03, pp. 6–10 (2003)

  8. “http://www.hawkeyeinnovations.co.uk/”

  9. Yu, X., Xu, C., Leong, H., Tian, Q., Tang, Q., Wan, K.: Trajectory-based ball detection and tracking with applications to semantic analysis of broadcast soccer video. In: Proceedings of ACM MultiMedia’ 03, pp. 11–20 (2003)

  10. Choi, S., Seo, Y., Kim, H., Hong, K.-S.: Where are the ball and players? Soccer game analysis with color-based tracking and image mosaick. In: Proceedings of ICIAP’97 (1997)

  11. Ohno, Y., Miura, J., Shirai, Y.: Tracking players and estimation of the 3d position of a ball in soccer games. In: The 2nd Asian Conference on Computer Vision Vol. 1, pp. 145–148 (2000)

  12. Bertini, M., Bimbo, A.D., Nunziati, W.: Player identification in soccer videos. In: Proceedings of ACM MultiMedia’04 workshop on Multimedia Information Retrieval (MIR), pp. 25–32 (2005)

  13. Assfalg, J., Bertini, M., Colombo, C., Bimbo, A., Nunziati, W.: Semantic annotation of soccer videos: automatic highlights identification. Comput. Vis. Image Understand. (CVIU) 92, 285–305 (2003)

    Article  Google Scholar 

  14. Zhang, D., Chang, S.-F.: Event detection in baseball video using superimposed caption recognition. In: Proceedings of ACM MultiMedia’02, pp. 315–318 (2002)

  15. Demiris, A., Diamantakos, G., Walczak, K., Reusens, E., Kerbiriou, P., Klein, K., Garcia, C., Marchal, I., Wingbermuhle, J., Boyle, E., Cellary, W., Ioannidis, N.: Piste: Mixed reality for sports tv. In: Proceedings of the International Workshop on Very Low Bitrate Video Coding (VLBV 01) (2001)

  16. Xie, L., Chang, S., Divakaran, A., Sun, H.: Unsupervised discovery of multilevel statistical video structures using hierarchical hidden markov models. In: Proceedings of IEEE ICME’03 (2003)

  17. Nitta, N., Babaguchi, N.: Automatic story segmentation of closed-caption text for semantic content analysis of broadcasted sports video. In: Proceedings of 8th International Workshop on Multimedia Information Systems ’02, pp. 110–116 (2002)

  18. Ekin, A., Tekalp, A.M., Mehrotra, R.: Automatic soccer video analysis and summarization. IEEE Trans. on Image Processing 12:7(5), 796–807 (2003)

    Article  Google Scholar 

  19. Assfalg, J., Bertini, M., Colombo, C., Nunziati, W.: Highlight extraction in soccer videos, Vol. 1. In: Proceedings of ICIAP’03, pp. 498–504 (2003)

  20. Rui, Y., Gupta, A., Acero, A.: Automatically extracting highlights for tv baseball programs. In: Proceedings of ACM MultiMedia’02, pp. 105–115 (2002)

  21. “http://viplab.dsi.unifi.it/assavid/”

  22. Assfalg, J., Bertini, M., Colombo, C., Bimbo, A.D., Nunziati, W.: Automatic extraction and annotation of soccer video highlights, Vol. 2. In: Proceedings of IEEE ICIP’2003, pp. 527–530 (2003)

  23. Han, M., Hua, W., Xu, W., Gong, Y.: An integrated baseball digest system using maximum entropy method. In: Proceedings of ACM MultiMedia’02, pp. 347–350 (2002)

  24. Sudhir, G., Lee, J.C.M., Jain, A.K.: Automatic classification of tennis video for high-level content-based retrieval. In: Proceedings of IEEE International Workshop on Content Based Access of Image and Video Database, pp. 81–90 (1998)

  25. Bertini, M., Bimbo, A.D., Cucchiara, R., Prati, A.: Semantic video adaptation based on automatic annotation of sport videos. In: Proceedings of ACM MultiMedia’04 workshop on Multimedia Information Retrieval (MIR), pp. 291–298 (2004)

  26. Wang, J., Xu, C., Chng, E., Wan, K., Tian, Q.: Automatic replay generation for soccer video broadcasting. In: Proceedings of ACM MultiMedia’04, pp. 31–38 (2004)

  27. Pingali, G., Jean, Y., Opalach, A., Carlbom, I.: Lucentvision: Converting real world events into multimedia experiences. In: Proceedings of IEEE ICME’00 (2000)

  28. Demiris, A., Traka, M., Reusens, E., Walczak, K., Garcia, C., Klein, K., Malerczyk, C., Kerbiriou, P., Bouville, C., Boyle, E., Ioannidis, N.: Enhanced sports broadcasting by means of augmented reality in mpeg-4. In: Proceedings of EUROIMAGE 2001; International Conference on Augmented, Virtual Environments and 3D Imaging (ICAV3D2001), pp. 10–13 (2001)

  29. Wan, K., Wang, J., Xu, C., Tian, Q.: Automatic sports highlights extraction with content augmentation. In: Proceedings of PCM’04, pp. 19–26 (2004)

  30. Bebie, T., Bieri, H.: Reconstructing soccer game from video sequence. In: Proceedings of IEEE ICIP’98, pp. 898–902 (1998)

  31. Koyama, T., Kitahara, I., Ohta, Y.: Live mixed-reality 3d video in soccer stadium. In: Proceedings of The 2nd IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR2003), pp. 178–187 (2003)

  32. Babaguchi, N., Nitta, N.: Intermodal collaboration: a strategy for semantic content analysis for broadcasted sports video, Vol. 1. In: Proceedings of IEEE ICIP’03, pp. 13–16 (2003)

  33. Xiong, Z., Radhakrishnan, R., Divakaran, A.: Generation of sports highlights using motion activity in combination with a common audio feature extraction framework, Vol. 1. In: Proceedings of ICIP’03, pp. 5–8 (2003)

  34. Hough, P.V.C.: Method and means for recognizing complex patterns. US Patent 3069654 (1962)

  35. Pilu, M.: On using raw mpeg motion vectors to determine global camera motion, Vol. 3309. In: Proceedings of SPIE VCIP’98, pp. 448–459 (1998)

  36. Tan, Y., Saur, D., Kulkami, S., Ramadge, P.: Rapid estimation of camera motion from compressed video with application to video annotation. IEEE Trans. Circ. Syst. Video Technol. (CSVT) 10(1), 133–146 (2000)

    Google Scholar 

  37. Wang, J., Xu, C., Chng, E., Tian, Q.: Sports highlight detection from keyword sequences using hmm. In: Proceedings of IEEE ICME’04 (2004)

  38. Hua, X., Lu, L., Zhang, H.: Ave: automated home video editing. In: Proceedings of ACM MultiMedia’03, pp. 490–497 (2003)

  39. Chin, J., Diehl, V., Norman, K.: Development of an instrument measuring user satisfaction of the human-computer interface. In: Proceedings of SIGCHI on Human Factors in Computing System, pp. 213–218 (1998)

  40. “http://www.ntu.edu.sg/home5/y020002/research/broadcasting/test.htm”

  41. Yang, M., Kriegman, D., Ahuja, N.: Detecting faces in images: a survey. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 24, 34–58 (2002)

    Article  Google Scholar 

  42. Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face recognition: a literature survey. ACM Comput. Surv. (CSUR) 35, 399–458 (2003)

    Article  Google Scholar 

  43. Gavrila, D.M.: The visual analysis of human movement: a survey. Comput. Vis. Image Understand. (CVIU) 73(1), 82–98 (1999)

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Infocomm Research, 21 Heng Mui Keng Terrace, Singapore, 119613, Singapore

    Jinjun Wang, Changsheng Xu & Qi Tian

  2. School of Computer Engineering, Nanyang Technological University, Singapore, Singapore

    Jinjun Wang & Engsiong Chng

  3. National Lab of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Hanqing Lu

Authors
  1. Jinjun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Changsheng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Engsiong Chng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hanqing Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qi Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Changsheng Xu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, J., Xu, C., Chng, E. et al. Automatic composition of broadcast sports video. Multimedia Systems 14, 179–193 (2008). https://doi.org/10.1007/s00530-008-0112-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-008-0112-6

Keywords

Search

Navigation