skip to main content
10.1145/3173574.3173890acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Extracting Regular FOV Shots from 360 Event Footage

Published: 21 April 2018 Publication History

Abstract

Video summaries are a popular way to share important events, but creating good summaries is hard. It requires expertise in both capturing and editing footage. While hiring a professional videographer is possible, this is too costly for most casual events. An alternative is to place 360 video cameras around an event space to capture footage passively and then extract regular field-of-view (RFOV) shots for the summary. This paper focuses on the problem of extracting such RFOV shots. Since we cannot actively control the cameras or the scene, it is hard to create "ideal' shots that adhere strictly to traditional cinematography rules. To better understand the tradeoffs, we study human preferences for static and moving camera RFOV shots generated from 360 footage. From the findings, we derive design guidelines. As a secondary contribution, we use these guidelines to develop automatic algorithms that we demonstrate in a prototype user interface for extracting RFOV shots from 360 videos.

Supplementary Material

suppl.mov (pn2956-file5.mp4)
Supplemental video

References

[1]
Arev, I., Park, H. S., Sheikh, Y., Hodgins, J., and Shamir, A. Automatic editing of footage from multiple social cameras. ACM Transactions on Graphics (TOG) 33, 4 (2014), 81.
[2]
Arijon, D. Grammar of the Film Language. Communication arts books. Hastings House, 1976.
[3]
Bianchi, M. Automatic video production of lectures using an intelligent and aware environment. In Proceedings of the 3rd international conference on Mobile and ubiquitous multimedia, ACM (2004), 117--123.
[4]
Cao, Z., Simon, T., Wei, S.-E., and Sheikh, Y. Realtime multi-person 2d pose estimation using part affinity fields. In CVPR (2017).
[5]
Carr, P., Mistry, M., and Matthews, I. Hybrid robotic/virtual pan-tilt-zom cameras for autonomous event recording. In Proceedings of the 21st ACM international conference on Multimedia, ACM (2013), 193--202.
[6]
Deselaers, T., Dreuw, P., and Ney, H. Pan, zoom, scantime-coherent, trained automatic video cropping. In Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, IEEE (2008), 1--8.
[7]
Fang, C., Lin, Z., Mech, R., and Shen, X. Automatic image cropping using visual composition, boundary simplicity and content preservation models. In Proceedings of the 22nd ACM international conference on Multimedia, ACM (2014), 1105--1108.
[8]
Fried, O., Shechtman, E., Goldman, D., and Finkelstein, A. Finding distractors in images. In Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on (2015), 1703--1712.
[9]
Gandhi, V., Ronfard, R., and Gleicher, M. Multi-clip video editing from a single viewpoint. In Proceedings of the 11th European Conference on Visual Media Production, ACM (2014), 9.
[10]
Gardner, M., Sunkavalli, K., Yumer, E., Shen, X., Gambaretto, E., Gagn´ e, C., and Lalonde, J. Learning to predict indoor illumination from a single image. CoRR abs/1704.00090 (2017).
[11]
He, L.-w., Cohen, M. F., and Salesin, D. H. The virtual cinematographer: a paradigm for automatic real-time camera control and directing. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, ACM (1996), 217--224.
[12]
Heck, R., Wallick, M., and Gleicher, M. Virtual videography. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 3, 1 (2007), 4.
[13]
Hu, H.-N., Lin, Y.-C., Liu, M.-Y., Cheng, H.-T., Chang, Y.-J., and Sun, M. Deep 360 pilot: Learning a deep agent for piloting through 360 sports videos. CVPR (2017).
[14]
Jain, E., Sheikh, Y., Shamir, A., and Hodgins, J. Gaze-driven video re-editing. ACM Transactions on Graphics (TOG) 34, 2 (2015), 21.
[15]
Lai, W.-S., Huang, Y., Joshi, N., Buehler, C., Yang, M.-H., and Kang, S. B. Semantic-driven generation of hyperlapse from 360 video. CVPR (2017).
[16]
Li, H., Lin, Z., Shen, X., Brandt, J., and Hua, G. A convolutional neural network cascade for face detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015), 5325--5334.
[17]
Liu, F., and Gleicher, M. Video retargeting: automating pan and scan. In Proceedings of the 14th ACM international conference on Multimedia, ACM (2006), 241--250.
[18]
Pavel, A., Hartmann, B., and Agrawala, M. Shot orientation controls for interactive cinematography with 360 video. In To appear, UIST'17, ACM (2017).
[19]
Saragih, J. M., Lucey, S., and Cohn, J. F. Face alignment through subspace constrained mean-shifts. In Computer Vision, 2009 IEEE 12th International Conference on, Ieee (2009), 1034--1041.
[20]
Su, Y.-C., and Grauman, K. Making 360 video watchable in 2d: Learning videography for click free viewing. CVPR (2017).
[21]
Wei, S.-E., Ramakrishna, V., Kanade, T., and Sheikh, Y. Convolutional pose machines. In CVPR (2016).
[22]
Zhang, J., Sclaroff, S., Lin, Z., Shen, X., Price, B., and Mech, R. Unconstrained salient object detection via proposal subset optimization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), 5733--5742.
[23]
Zhang, L., Song, M., Yang, Y., Zhao, Q., Zhao, C., and Sebe, N. Weakly supervised photo cropping. IEEE Transactions on Multimedia 16, 1 (2014), 94--107.

Cited By

View all
  • (2022)Synthesis-Assisted Video Prototyping From a DocumentProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545676(1-10)Online publication date: 29-Oct-2022
  • (2021)Automatic Instructional Video Creation from a Markdown-Formatted TutorialThe 34th Annual ACM Symposium on User Interface Software and Technology10.1145/3472749.3474778(677-690)Online publication date: 10-Oct-2021
  • (2021)Diagnosing Bias in the Gender Representation of HCI Research Participants: How it Happens and Where We AreProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445383(1-18)Online publication date: 6-May-2021
  • Show More Cited By

Index Terms

  1. Extracting Regular FOV Shots from 360 Event Footage

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
    April 2018
    8489 pages
    ISBN:9781450356206
    DOI:10.1145/3173574
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 21 April 2018

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. 360 video
    2. event video
    3. video editing
    4. video summaries

    Qualifiers

    • Research-article

    Conference

    CHI '18
    Sponsor:

    Acceptance Rates

    CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;
    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)23
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)Synthesis-Assisted Video Prototyping From a DocumentProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545676(1-10)Online publication date: 29-Oct-2022
    • (2021)Automatic Instructional Video Creation from a Markdown-Formatted TutorialThe 34th Annual ACM Symposium on User Interface Software and Technology10.1145/3472749.3474778(677-690)Online publication date: 10-Oct-2021
    • (2021)Diagnosing Bias in the Gender Representation of HCI Research Participants: How it Happens and Where We AreProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445383(1-18)Online publication date: 6-May-2021
    • (2020)Automatic Video Creation From a Web PageProceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology10.1145/3379337.3415814(279-292)Online publication date: 20-Oct-2020
    • (2019)A Tool for Navigating and Editing 360 Video of Social Conversations into Shareable HighlightsProceedings of the 45th Graphics Interface Conference10.20380/GI2019.14(1-9)Online publication date: 1-Jun-2019
    • (2019)Automatic Camera Path Generation from 360 VideoAdvances in Visual Computing10.1007/978-3-030-33720-9_39(505-514)Online publication date: 7-Oct-2019

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media