Skip to main content
Springer Nature Link
Log in

Free-Viewpoint Video for TV Sport Production

  • Chapter
  • First Online:
Image and Geometry Processing for 3-D Cinematography

Part of the book series: Geometry and Computing ((GC,volume 5))

  • 1480 Accesses

Abstract

Free-viewpoint video in sports TV production presents a challenging problem involving the conflicting requirements of broadcast picture quality with video-rate generation of novel views, together with practical problems in developing robust systems for cost effective deployment at live events.To date most multiple view video systems have been developed for studio applications with a fixed capture volume, controlled illumination and backgrounds. Live outdoor events such as sports present a number of additional challenges for both acquisition and processing. Multiple view capture systems in sports such as football must cover the action taking place over an entire pitch with video acquisition at sufficient resolution for analysis and production of desired virtual camera views.In this chapter we identify the requirements for broadcast production of free-viewpoint video in sports and review the state-of-the-art. We present the iview free-viewpoint video system which enables production of novel camera views of sports action for use in match commentary, for example the referees viewpoint. Automatic online calibration, segmentation and reconstruction is performed to allow rendering of novel viewpoints from the moving match cameras.Results are reported of production trials in football (soccer) and rugby which demonstrate free-viewpoint video with a visual quality comparable to the broadcast footage.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., Thrun, S.: Performance capture from sparse multi-view video. In: Proc. ACM SIGGRAPH, 27(3) (2008)

    Google Scholar 

  2. Boykov, Y., Veksle, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)

    Article  Google Scholar 

  3. Broadhurst, A., Drummond, T.W., Cipolla, R.: A probabilistic framework for space carving. In: International Conf. on Computer Vision, pp. 388–393 (2001)

    Google Scholar 

  4. Carranza, J., Theobalt, C., Magnor, M., Seidel, H.-P.: Free-viewpoint video of human actors. In: Proc. ACM SIGGRAPH, pp. 565–577 (2003)

    Google Scholar 

  5. Chen, S.E., Williams, L.: View interpolation for image synthesis. In: Proc. ACM SIGGRAPH (1993)

    Google Scholar 

  6. Connor, K., Reid, I.: A multiple view layered representation for dynamic novel view synthesis. In: British Machine Vision Conference (2003)

    Google Scholar 

  7. Debevec, P., Yu, Y., Borshukov, G.: Efficient view-dependent image-based rendering with projective texture-mapping. In: 9th Eurographics Rendering Workshop, pp. 105–116 (1998)

    Google Scholar 

  8. Deutscher, J., Davidson, A., Reid, I.: Automatic partitioning of high-dimensional search spaces associated with articulated body motion capture. In: Conference on Computer Vision and Pattern Recognition, pp. 669–676 (2001)

    Google Scholar 

  9. Eisemann, M., Decker, B., Magnor, M.A., Bekaert, P., Aguiar, E., Ahmed, N., Theobalt, C., Sellent, A.: Floating textures. Comput. Graph. Forum 27(2), 409–418 (2006)

    Article  Google Scholar 

  10. Franco, F.S., Boyer, E.: Exact polyhedral visual hulls. In: British Machine Vision Conference, pp. 329–338 (2003)

    Google Scholar 

  11. Franco, F.S., Boyer, E.: Fusion of multi-view silhouette cues using a space occupacy grid. In: International Conf. on Computer Vision, pp. 1747–1753 (2005)

    Google Scholar 

  12. Franco, F.S., Menier, C., Boyer, E., Raffin, B.: A distributed approach for real-time 3D modeling. In: CVPR Workshop on Real-Time 3D Sensors and their Applications (2004)

    Google Scholar 

  13. Goldluecke, B., Magnor, M.: Space–time isosurface evolution for temporally coherent 3D reconstruction. In: Conference on Computer Vision and Pattern Recognition, pp. S–E, Washington, DC, USA, July (2004). IEEE Computer Society.

    Google Scholar 

  14. Grau, O., Easterbrook, J.: Effects of camera aperture correction on keying of broadcast video. In: Proc. of the 5rd European Conference on Visual Media Production (CVMP) (2008)

    Google Scholar 

  15. Grau, O., Hilton, A., Kilner, J., Miller, G., Sargeant, T., Starck, J.: A free-viewpoint video system for visualisation of sports scenes. SMPTE Motion Imag. J. 116(5–6), 213–219 (2007)

    Google Scholar 

  16. Grau, O., Prior-Jones, M., Thomas, G.: 3D modelling and rendering of studio and sport scenes for TV applications. In: Proc. of WIAMIS (2005)

    Google Scholar 

  17. Grau, O., Thomas, G.A., Hilton, A., Kilner, J., Starck, J.: A robust free-viewpoint video system for sport scenes. In: Proceeding of 3DTV conference 2007, Kos, Greece (2007)

    Google Scholar 

  18. Guillemaut, J.-Y., Kilner, J., Hilton, A.: Robust graph-cut scene segmentation and reconstruction for free-viewpoint video of complex dynamic scenes. In: IEEE Int.Conf. on Computer Vision, ICCV (2009)

    Google Scholar 

  19. Guillemaut, J.-Y., Kilner, J., Starck, J., Hilton, A.: Dynamic feathering: minimising blending artefacts in view dependent rendering. In: IET European Conference on Visual Media Production, pp. 1–8 (2007)

    Google Scholar 

  20. Guillemaut, J.Y., Hilton, A., Starck, J., Kilner, J.J., Grau, O.: A Baysian framework for simultaneous reconstruction and matting. In: IEEE Int.Conf. on 3D Imaging and Modeling (2007)

    Google Scholar 

  21. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)

    MATH  Google Scholar 

  22. Hillman, P., Hannah, J., Renshaw, D.: Foreground/background segmentation of motion picture images and image sequences. IEE Trans. Vis. Image Signal Process. 152(4), 387–397 (2005)

    Article  Google Scholar 

  23. Inamoto, N., Saito, H.: Virtual viewpoint replay for a soccer match by view interpolation from multiple cameras. IEEE Trans. Multimedia 9(6), 1155–1166 (2007)

    Article  Google Scholar 

  24. Irani, M., Hassner, T., Anandan, P.: What does the scene look like from a scene point? In: European Conference on Computer Vision (2002)

    Google Scholar 

  25. Kanade, T., Rander, P.: Virtualized reality: Constructing virtual worlds from real scenes. IEEE MultiMedia 4(2), 34–47 (1997)

    Article  Google Scholar 

  26. Kanade, T., Rander, P.W., Narayanan, P.J.: Virtualized reality: constructing virtual worlds from real scenes. IEEE MultiMedia 4(1), 34–47 (1997)

    Article  Google Scholar 

  27. Kilner, J.J., Starck, J., Hilton, A., Guillemaut, J.Y., Grau, O.: Dual mode deformable models for free-viewpoint video of outdoor sports events. In: IEEE Int. Conf. on 3D Imaging and Modeling (2007)

    Google Scholar 

  28. Kimura, K., Saito, H.: Player viewpoint video synthesis using multiple cameras. In: IEE European Conference on Visual Media Production, pp. 112–121 (2005)

    Google Scholar 

  29. Laurentini, A.: The visual hull concept for silhouette based image understanding. IEEE Trans. Pattern Anal. Mach. Intell. 16(2), 150–162 (1994)

    Article  Google Scholar 

  30. Matusik, W., Buehler, C., Raskar, R., Gortler, S.J., McMillan, L.: Image-based visual hulls. In: Proceedings of ACM SIGGRAPH, pp. 369–374 (2000)

    Google Scholar 

  31. Miller, G., Hilton, A., Starck, J.: Interactive free-viewpoint video. In: IEE European Conf. on Visual Media Production, pp. 50–59 (2005)

    Google Scholar 

  32. Miller, G., Starck, J.R., Hilton, A.: Projective surface refinement for free-viewpoint video. In: IET European Conference on Visual Media Production, pp. 153–162 (2006)

    Google Scholar 

  33. Moeslund, T., Hilton, A., Kruger, V.: A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst. 104(2–3), 90–127 (2006)

    Article  Google Scholar 

  34. Moezzi, S., Katkere, A., Kuramura, D.Y., Jain, R.: Reality modeling and visualization from multiple video sequences. IEEE Comput. Graph. Appl. 16(6), 58–63 (1996)

    Article  Google Scholar 

  35. Moezzi, S., Tai, L.C., Gerard, P.: Virtual view generation for 3D digital video. IEEE MultiMedia 4(1), 18–25 (1997)

    Article  Google Scholar 

  36. Scarstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1), 7–42 (2002)

    Article  Google Scholar 

  37. Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: Conference on Computer Vision and Pattern Recognition, pp. 519–528 (2006)

    Google Scholar 

  38. Seitz, S.M., Dyer, C.R.: View morphing: synthesizing 3D metamorphosis using image transforms. In: Proc. ACM SIGGRAPH, pp. 21–30 (1996)

    Google Scholar 

  39. Seitz, S.M., Dyer, C.R.: Photorealistic scene reconstruction by voxel coloring. Int. J. Comput. Vis. 35(2), 1–23 (1997)

    Google Scholar 

  40. Shum, H.-Y., Kang, S.B., Chan, S.-C.: Survey of image-based representations and compression techniques. IEEE Trans. Circuits Syst. Video Technol. 13(11) (2003)

    Google Scholar 

  41. Slabaugh, G., Culbertson, W.B., Malzbender, T., Stevens, M., Schafer, R.: A survey of methods for volumetric scene reconstruction from photographs. In: Volume Graphics (2001)

    Google Scholar 

  42. Starck, J., Hilton, A.: Virtual view synthesis of people from multiple view video. Graph. Models 67(6), 600–620 (2005)

    Article  MATH  Google Scholar 

  43. Starck, J., Hilton, A.: Surface capture for performance-based animation. IEEE Comput. Graph. Appl. 27(3), 21–31 (2007)

    Article  Google Scholar 

  44. Stich, T., Linz, C., Wallraven, C., Cunningham, D., Magnor, M.: Perception-motivated interpolation of image sequences. In: Proc. ACM APGV, pp. 97–106 (2008)

    Google Scholar 

  45. Thomas, G.A.: Real-time camera pose estimation for augmenting sports scenes. In: European Conference on Visual Media Production, pp. 10–19 (2006)

    Google Scholar 

  46. Vedula, S., Baker, S., Rander, P., Collins, R., Kanade, T.: Three-dimensional scene flow. IEEE Trans. Pattern Anal. Mach. Intell. 27(3), 475–480 (2005)

    Article  Google Scholar 

  47. Vlasic, D., Baran, I., Matusik, W., Popović, J.: Articulated mesh animation from multi-view silhouettes. In: Proc. ACM SIGGRAPH, pp. 1–9 (2008)

    Google Scholar 

  48. Zitnick, C.L., Kang, S.B., Uyttendaele, M., Winder, S., Szeliski, R.: High-quality video view interpolation using a layered representation. In: Proc. ACM SIGGRAPH, pp. 600–608 (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Surrey, Surrey, UK

    Adrian Hilton, Jean-Yves Guillemaut & Joe Kilner

  2. BBC R&D, London, UK

    Oliver Grau & Graham Thomas

Authors
  1. Adrian Hilton
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jean-Yves Guillemaut
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Joe Kilner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Oliver Grau
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Graham Thomas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adrian Hilton .

Editor information

Editors and Affiliations

  1. LEAR Team, INRIA, avenue de l'Europe, Montbonnot, 38330, France

    Rémi Ronfard

  2. Div. Engineering, Brown University, Providence, 02912, USA

    Gabriel Taubin

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Hilton, A., Guillemaut, JY., Kilner, J., Grau, O., Thomas, G. (2010). Free-Viewpoint Video for TV Sport Production. In: Ronfard, R., Taubin, G. (eds) Image and Geometry Processing for 3-D Cinematography. Geometry and Computing, vol 5. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12392-4_4

Download citation

Publish with us

Policies and ethics