skip to main content
10.1145/3505284.3532975acmconferencesArticle/Chapter ViewAbstractPublication PagesimxConference Proceedingsconference-collections
Work in Progress

Classification of the Video Type and Device Used in 360-Degree Videos from the Trajectories of its Viewers’ Orientations with LSTM Neural Network Models

Authors Info & Claims
Published:22 June 2022Publication History

ABSTRACT

360-degree videos are consumed in diverse devices: some based in immersive interfaces, such as viewed through Virtual Reality headsets and some based in non-immersive interfaces, as in a computer with a pointing device or mobile devices with touchscreens. We have found, in prior work, significant differences in user behavior between these devices. From a dataset of the trajectories of the users’ head orientation in 775 video reproductions, we classify which kind of video was played (two values) and which of the four possible devices was used to reproduce these videos. We found that recurrent neural network models based on LSTM layers are able to classify the video type and the device used to play the video with an average accuracy of over 90% with only four seconds of trajectory. We are convinced that this knowledge can improve techniques to predict future viewports used in viewport-adaptive streaming when diverse devices are used.

References

  1. Yanan Bao, Huasen Wu, Tianxiao Zhang, Albara Ah Ramli, and Xin Liu. 2016. Shooting a Moving Target: Motion-Prediction-Based Transmission for 360-Degree Videos. In 2016 IEEE International Conference on Big Data, Big Data 2016, 1161–1170. https://doi.org/10.1109/BigData.2016.7840720Google ScholarGoogle ScholarCross RefCross Ref
  2. Ching Ling Fan, Jean Lee, Wen Chih Lo, Chun Ying Huang, Kuan Ta Chen, and Cheng Hsin Hsu. 2017. Fixation prediction for 360° video streaming in head-mounted virtual reality. In Proceedings of the 27th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, NOSSDAV 2017, 67–72. https://doi.org/10.1145/3083165.3083180Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Thierry Fautier. 2016. VR video ecosystem for live distribution. In IBC 2016 Conference, 28 (13 .)-28 (13 .). https://doi.org/10.1049/ibc.2016.0028Google ScholarGoogle ScholarCross RefCross Ref
  4. Felix A. Gers and Jurgen Schmidhuber. 2000. Recurrent nets that time and count. In Proceedings of the International Joint Conference on Neural Networks, 189–194. https://doi.org/10.1109/ijcnn.2000.861302Google ScholarGoogle Scholar
  5. Felix A. Gers, Jürgen Schmidhuber, and Fred Cummins. 2000. Learning to forget: Continual prediction with LSTM. Neural Computation 12, 10: 2451–2471. https://doi.org/10.1162/089976600300015015Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation 9, 8: 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Evgeny Kuzyakov and David Pio. 2016. Next-generation video encoding techniques for 360 video and VR. Facebook Code. Retrieved from https://code.facebook.com/posts/1126354007399553/next-generation-video-encoding-techniques-for-360-video-and-vr/Google ScholarGoogle Scholar
  8. Ed Lantz, Steve Bryson, David Zeltzer, Mark T. Bolas, Bertrand de La Chapelle, and David Bennett. 1996. Future of virtual reality: Head mounted displays versus spatially immersive displays. In Proceedings of the ACM SIGGRAPH Conference on Computer Graphics. https://doi.org/10.1145/237170.237289Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Ulrich Neumann, Thomas Pintaric, and Albert Rizzo. 2000. Immersive panoramic video. In Proceedings of the eighth ACM international conference on Multimedia – MULTIMEDIA ’00, 493–494. https://doi.org/10.1145/354384.376408Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Antoni Oliver, Javier del Molino, and Antoni Bibiloni Coll. 2020. Palma360: A 360-degree Web-based Multi-device Hypervideo. In IMX ’20: ACM International Conference on Interactive Media Experiences, 5. https://doi.org/10.1145/3391614.3399394Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Wendy Powell, Vaughan Powell, Phillip Brown, Marc Cook, and Jahangir Uddin. 2016. Getting around in google cardboard – exploring navigation preferences with low-cost mobile VR. In 2016 IEEE 2nd Workshop on Everyday Virtual Reality (WEVR), 5–8. https://doi.org/10.1109/WEVR.2016.7859536Google ScholarGoogle ScholarCross RefCross Ref

Recommendations

Comments

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Sign in
  • Published in

    cover image ACM Conferences
    IMX '22: Proceedings of the 2022 ACM International Conference on Interactive Media Experiences
    June 2022
    390 pages
    ISBN:9781450392129
    DOI:10.1145/3505284

    Copyright © 2022 Owner/Author

    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    • Published: 22 June 2022

    Check for updates

    Qualifiers

    • Work in Progress
    • Research
    • Refereed limited

    Acceptance Rates

    Overall Acceptance Rate69of245submissions,28%

    Upcoming Conference

    IMX '24
  • Article Metrics

    • Downloads (Last 12 months)43
    • Downloads (Last 6 weeks)16

    Other Metrics

PDF Format

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format .

View HTML Format