skip to main content
research-article

The Cameraman Operating My Virtual Camera is Artificial: Can the Machine Be as Good as a Human?

Published: 02 June 2015 Publication History

Abstract

In this article, we argue that the energy spent in designing autonomous camera control systems is not spent in vain. We present a real-time virtual camera system that can create smooth camera motion. Similar systems are frequently benchmarked with the human operator as the best possible reference; however, we avoid a priori assumptions in our evaluations. Our main question is simply whether we can design algorithms to steer a virtual camera that can compete with the user experience for recordings from an expert operator with several years of experience? In this respect, we present two low-complexity servoing methods that are explored in two user studies. The results from the user studies give a promising answer to the question pursued. Furthermore, all components of the system meet the real-time requirements on commodity hardware. The growing capabilities of both hardware and network in mobile devices give us hope that this system can be deployed to mobile users in the near future. Moreover, the design of the presented system takes into account that services to concurrent users must be supported.

References

[1]
Adel Ahmed and Peter Eades. 2005. Automatic camera path generation for graph navigation in 3D. In Proceedings of the Asia-Pacific Symposium on Information Visualisation. 27--32. http://dl.acm.org/citation.cfm?id&equal;1082315.1082320
[2]
Y. Ariki, S. Kubota, and M. Kumano. 2006. Automatic production system of soccer sports video by digital camera work based on situation recognition. In Proceedings of the IEEE International Symposium on Multimedia. 851--860.
[3]
Peter Carr and Richard Hartley. 2009. Portable multi-megapixel camera with real-time recording and playback. In Proceedings of the Conference on Digital Image Computing: Techniques and Applications. 74--80.
[4]
Peter Carr, Michael Mistry, and Iain Matthews. 2013. Hybrid robotic/virtual pan-tilt-zom cameras for autonomous event recording. In Proceedings of the ACM Multimedia Conference. 193--202.
[5]
Joel Carranza, Christian Theobalt, Marcus A. Magnor, and Hans-Peter Seidel. 2003. Free viewpoint video of human actors. ACM Trans. Graph. 22, 3, 569--577.
[6]
Fan Chen and Christophe De Vleeschouwer. 2010. Personalized production of basketball videos from multisensored data under limited display resolution. Computer Vision Image Understanding 114, 6, 667--680.
[7]
Kuan-Ta Chen, Chen-Chi Wu, Yu-Chun Chang, and Chin-Laung Lei. 2009. A crowd-sourceable QoE evaluation framework for multimedia content. In Proceedings of the ACM Multimedia Conference. 491--500.
[8]
Shenchang Eric Chen. 1995. QuickTime VR: An image-based approach to virtual environment navigation. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 29--38.
[9]
Marc Christie, Rumesh Machap, Jean-Marie Normand, Patrick Olivier, and Jonathan Pickering. 2005. Virtual camera planning: A survey. In Smart Graphics, Lecture Notes in Computer Science, vol. 3638, 40--52. 4
[10]
A Dearden, Y Demiris, and O Grau. 2007. Learning models of camera control for imitation in football matches. In Proceedings of the Artificial and Ambient Intelligence Symposium. 227--231.
[11]
Paul E. Debevec, Camillo J. Taylor, and Jitendra Malik. 1996. Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'96). ACM, New York, 11--20.
[12]
Myléne C. Q. Farias, John M. Foley, and Sanjit K. Mitra. 2007. Detectability and annoyance of synthetic blocky, blurry, noisy, and ringing artifacts. IEEE Trans. Signal Process. 55, 6, 2954--2964.
[13]
Christoph Fehn, Christian Weissig, Ingo Feldmann, Markus Muller, Peter Eisert, Peter Kauff, and Hans Bloss. 2006. Creation of high-resolution video panoramas of sport events. In Proceedings of the IEEE International Symposium on Multimedia. 291--298.
[14]
Eric Foote, Peter Carr, Patrick Lucey, Yaser Sheikh, and Iain Matthews. 2013. One-man-band: A touch screen interface for producing live multi-camera sports broadcasts. In Proceedings of the ACM Multimedia Conference. 163--172.
[15]
Vamsidhar Reddy Gaddam, Carsten Griwodz, and Påal Halvorsen. 2014a. Automatic exposure for panoramic systems in uncontrolled lighting conditions: a football stadium case study. In Proceedings of SPIE: The Engineering Reality of Virtual Reality. 90120C--90120C--9.
[16]
Vamsidhar Reddy Gaddam, Ragnar Langseth, Sigurd Ljødal, Pierre Gurdjos, Vincent Charvillat, Carsten Griwodz, and Påal Halvorsen. 2014b. Interactive Zoom and Panning from Live Panoramic Video. In Proceedings of the ACM International Workshop on Network and Operating Systems Support for Digital Audio and Video. Article 19.
[17]
Lutz Goldmann, Francesca De Simone, Frederic Dufaux, Touradj Ebrahimi, Rudolf Tanner, and Mauro Lattuada. 2010. Impact of video transcoding artifacts on the subjective quality. In Proceedings of the International Workshop on Quality of Multimedia Experience. 52--57.
[18]
Patrik Goorts, Steven Maesen, Maarten Dumont, Sammy Rogmans, and Philippe Bekaert. 2014. Free viewpoint video for soccer using histogram-based validity maps in plane sweeping. In Proceedings of the International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. 378--386.
[19]
O. Grau, T. Pullen, and G. A. Thomas. 2004. A combined studio production system for 3-D capturing of live action and immersive actor feedback. IEEE Trans. Circuits Syst. Video Technol. 14, 3, 370--380.
[20]
O. Grau, G. A. Thomas, A. Hilton, J. Kilner, and J. Starck. 2007. A robust free-viewpoint video system for sport scenes. In Proceedings of the 3DTV Conference. 1--4.
[21]
Påal Halvorsen, Simen Såegrov, Asgeir Mortensen, David K. C. Kristensen, Alexander Eichhorn, Magnus Stenhaug, Stian Dahl, Håakon Kvale Stensland, Vamsidhar Reddy Gaddam, Carsten Griwodz, and Dag Johansen. 2013. BAGADUS: An Integrated system for arena sports analytics -- A soccer case study. In Proceedings of the ACM Multimedia Conference. 48--59.
[22]
S. Hutchinson, G. D. Hager, and P. I. Corke. 1996. A tutorial on visual servo control. IEEE Trans. Rob. Automation 12, 5, 651--670.
[23]
ITU-R. 2002. BT.500-11. Methodology for the subjective assessment of the quality of television pictures. https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.500-11-200206-SIIPDF-E.pdf.
[24]
ITU-T. 1998. P.911. Subjective audiovisual quality assessment methods for multimedia applications. https://www.itu.int/rec/T-REC-P.911-199812-1/en.
[25]
Michael Jenkin, James Elder, and Greg Pintilie. 1998. Loosely-coupled telepresence through the panoramic image server. In Vision Interface: Real World Applications of Computer Vision.
[26]
R. Kaiser, M. Thaler, A. Kriechbaum, H. Fassold, W. Bailer, and J. Rosner. 2011. Real-time person tracking in high-resolution panoramic video for automated broadcast production. In Proceedings of the European Conference on Visual Media Production. 21--29.
[27]
Takeo Kanade, Peter Rander, and P. J. Narayanan. 1997. Virtualized reality: Constructing virtual worlds from real scenes. IEEE MultiMedia 4, 1, 34--47.
[28]
Jong-Seok Lee, Lutz Goldmann, and Touradj Ebrahimi. 2012. Paired comparison-based subjective quality assessment of stereoscopic images. Multimedia Tools Appl. 67, 1, 31--48.
[29]
Christian Lipski, Christian Linz, Kai Berger, and Marcus Magnor. 2009. Virtual video camera: Image-based viewpoint navigation through space and time. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. Article 93.
[30]
Aditya Mavlankar and Bernd Girod. 2010. Video streaming with interactive pan/tilt/zoom. In High-Quality Visual Experience, Marta Mrak, Mislav Grgic, and Murat Kunt (Eds.), 431--455. 19
[31]
Pengpeng Ni, Ragnhild Eg, Alexander Eichhorn, Carsten Griwodz, and Påal Halvorsen. 2011. Flicker effects in adaptive video streaming to handheld devices. In Proceedings of the ACM Multimedia Conference. 463--472.
[32]
N. Papadakis, A. Baeza, I. Rius, X. Armangue, A. Bugeau, O. D'Hondt, P. Gargallo, V. Caselles, and S. Sagas. 2010. Virtual camera synthesis for soccer game replays. In Proceedings of the Conference on Visual Media Production. 97--106.
[33]
Jinchang Ren, Ming Xu, James Orwell, and GraemeA. Jones. 2010. Multi-camera video surveillance for real-time analysis and reconstruction of soccer games. Machine Vision Appl. 21, 6, 855--863.
[34]
Xinding Sun, J. Foote, D. Kimber, and B. S. Manjunath. 2005. Region of interest extraction and virtual camera control based on panoramic video capturing. IEEE Trans. Multimedia 7, 5, 981--990.
[35]
Marius Tennøe, Espen Helgedagsrud, Mikkel Nåess, Henrik Kjus Alstad, Håakon Kvale Stensland, Vamsidhar Reddy Gaddam, Dag Johansen, Carsten Griwodz, and Påal Halvorsen. 2013. Efficient implementation and processing of a real-time panorama video pipeline. In Proceedings of the IEEE International Symposium on Multimedia.
[36]
Jinjun Wang, Changsheng Xu, Engsiong Chng, Kongwah Wah, and Qi Tian. 2004. Automatic replay generation for soccer video broadcasting. In Proceedings of the ACM Multimedia Conference. 32--39.
[37]
Wanmin Wu, Ahsan Arefin, Raoul Rivas, Klara Nahrstedt, Renata M. Sheppard, and Zhenyu Yang. 2009. Quality of experience in distributed interactive multimedia environments: Toward a theoretical framework. In Proceedings of the ACM Multimedia Conference. 481--490.
[38]
M. Xu, J. Orwell, L. Lowey, and D. Thirde. 2005. Architecture and algorithms for tracking football players with multiple cameras. In IEE Proc. Vision Image Signal Process. 152, 2, 232--241.
[39]
Wei Xu and Jane Mulligan. 2013. Panoramic video stitching from commodity HDTV cameras. Multimedia Systems 19, 5, 407--426.
[40]
T. Yokoi and H. Fujiyoshi. 2005. Virtual camerawork for generating lecture video from high resolution images. In Proceedings of the IEEE International Conference on Multimedia and Expo.
[41]
Xinguo Yu, Changsheng Xu, Hon Wai Leong, Qi Tian, Qing Tang, and Kong Wah Wan. 2003. Trajectory-based ball detection and tracking with applications to semantic analysis of broadcast soccer video. In Proceedings of the ACM Multimedia Conference. 11--20.

Cited By

View all
  • (2022)Automating sports broadcasting using ultra-high definition cameras, neural networks, and classical denoisingApplications of Digital Image Processing XLV10.1117/12.2633075(36)Online publication date: 3-Oct-2022
  • (2022)Automatic football video production system with edge processingMachine Vision and Applications10.1007/s00138-022-01283-033:2Online publication date: 21-Feb-2022
  • (2021)Automated Event Detection and Classification in Soccer: The Potential of Using Multiple ModalitiesMachine Learning and Knowledge Extraction10.3390/make30400513:4(1030-1054)Online publication date: 16-Dec-2021
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications
ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 11, Issue 4
April 2015
231 pages
ISSN:1551-6857
EISSN:1551-6865
DOI:10.1145/2788342
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 June 2015
Accepted: 01 February 2015
Revised: 01 November 2014
Received: 01 July 2014
Published in TOMM Volume 11, Issue 4

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Interactive immersion
  2. panning
  3. panorama video
  4. quality of experience
  5. real-time
  6. user studies
  7. virtual camera
  8. visual servoing
  9. zoom

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

  • Norwegian Research Council

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)7
  • Downloads (Last 6 weeks)1
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Automating sports broadcasting using ultra-high definition cameras, neural networks, and classical denoisingApplications of Digital Image Processing XLV10.1117/12.2633075(36)Online publication date: 3-Oct-2022
  • (2022)Automatic football video production system with edge processingMachine Vision and Applications10.1007/s00138-022-01283-033:2Online publication date: 21-Feb-2022
  • (2021)Automated Event Detection and Classification in Soccer: The Potential of Using Multiple ModalitiesMachine Learning and Knowledge Extraction10.3390/make30400513:4(1030-1054)Online publication date: 16-Dec-2021
  • (2021)Context-based camera selection from multiple video streamsMultimedia Tools and Applications10.1007/s11042-021-11674-6Online publication date: 5-Nov-2021
  • (2020)As Seen on TV: Automatic Basketball Video Production using Gaussian-based Actionness and Game States Recognition2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW50498.2020.00455(3911-3920)Online publication date: Jun-2020
  • (2019)Keep Your Eye on the Puck: Automatic Hockey Videography2019 IEEE Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV.2019.00179(1636-1644)Online publication date: Jan-2019
  • (2018)Quantified Soccer Using Positional Data: A Case StudyFrontiers in Physiology10.3389/fphys.2018.008669Online publication date: 6-Jul-2018
  • (2017)Where should cameras look at soccer games: Improving smoothness using the overlapped hidden Markov modelComputer Vision and Image Understanding10.1016/j.cviu.2016.10.017159(59-73)Online publication date: Jun-2017
  • (2016)Semi-Automatic Camera and Switcher Control for Live BroadcastProceedings of the ACM International Conference on Interactive Experiences for TV and Online Video10.1145/2932206.2933559(129-134)Online publication date: 17-Jun-2016
  • (2016)Learning Online Smooth Predictors for Realtime Camera Planning Using Recurrent Decision Trees2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR.2016.507(4688-4696)Online publication date: Jun-2016

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media