skip to main content
10.1145/2393347.2396449acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
abstract

Interactive music video application for smartphones based on free-viewpoint video and audio rendering

Published: 29 October 2012 Publication History

Abstract

This paper presents a novel interactive music video application for smartphones based on free-viewpoint video technology in conjunction with three-dimensional positional audio technology. A user can enjoy a music video from a moving viewpoint that the user can manipulate by the touch screen, with the positional audio through the headphone. A user can even manipulate the positions of the performers on the stage as well as the viewpoint. The application, consisting of our audio rendering engine for multiple AAC ADTS files and our video rendering engine for multiple H.264 ES files, runs on a smartphone in stand-alone mode. The application has been released as official content from a music label for Android and iOS.

References

[1]
J. Blauert. Spatial Hearing: The Psychophysics of Human Sound Localization. The MIT Press, Cambridge, Massachusetts, 1996.
[2]
T. Fukuda, T. Horiuchi, H. Hokari, and S. Shimada. Relative distance perception by manipulating ILD of HRTFs. Acoustic Science and Technology edited by the Acoustical Society of Japan, 24(5):325--326, 2003.
[3]
M. Germann, A. Hornung, R. Keiser, R. Ziegler, S. Würmlin, and M. Gross. Articulated billboards for video-based rendering. Eurographics, 29(2):585--594, 2010.
[4]
A. Ishikawa, M. P. Tehrani, S. Naito, S. Sakazawa, and A. Koike. Free viewpoint video generation for walk-through experience using image-based rendering. In ACM International Conference on Multimedia, pages 1007--1008, 2008.
[5]
T. Kanade, P. Rander, and P. J. Narayanan. Virtualized reality: Constructing virtual worlds from real scenes. IEEE Multimedia, 4(1):34--47, 1997.
[6]
M. P. Tehrani, K. Niwa, N. Fukushima, Y. Hirano, T. Fujii, M. Tanimoto, K. Takeda, K. Mase, A. Ishikawa, S. Sakazawa, and A. Koike. 3DAV integrated system featuring arbitrary listening-point and viewpoint generation. In IEEE Workshop on Multimedia Signal Processing, pages 855--860, 2008.

Cited By

View all
  • (2023)OtonoVR—Particular Spatial Focused 360-degree Video Player App: The Development and Practical Deployment特定空間フォーカス型360度動画再生アプリ「音のVR」の開発と実用化The Journal of The Institute of Image Information and Television Engineers10.3169/itej.77.38877:3(388-393)Online publication date: 2023
  • (2023)Augmented Immersive Viewing and Listening Experience Based on Arbitrarily Angled Interactive Audiovisual RepresentationProceedings of the 25th International Conference on Multimodal Interaction10.1145/3577190.3614138(79-83)Online publication date: 9-Oct-2023
  • (2023)App interaction design in the pop music singing teaching system based on differential evolution algorithmSoft Computing10.1007/s00500-023-08167-z27:13(9065-9075)Online publication date: 20-Apr-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MM '12: Proceedings of the 20th ACM international conference on Multimedia
October 2012
1584 pages
ISBN:9781450310895
DOI:10.1145/2393347

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 October 2012

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. free-viewpoint video
  2. head-related transfer function
  3. smartphone
  4. tablet
  5. three-dimensional positional audio

Qualifiers

  • Abstract

Conference

MM '12
Sponsor:
MM '12: ACM Multimedia Conference
October 29 - November 2, 2012
Nara, Japan

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)8
  • Downloads (Last 6 weeks)0
Reflects downloads up to 26 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2023)OtonoVR—Particular Spatial Focused 360-degree Video Player App: The Development and Practical Deployment特定空間フォーカス型360度動画再生アプリ「音のVR」の開発と実用化The Journal of The Institute of Image Information and Television Engineers10.3169/itej.77.38877:3(388-393)Online publication date: 2023
  • (2023)Augmented Immersive Viewing and Listening Experience Based on Arbitrarily Angled Interactive Audiovisual RepresentationProceedings of the 25th International Conference on Multimodal Interaction10.1145/3577190.3614138(79-83)Online publication date: 9-Oct-2023
  • (2023)App interaction design in the pop music singing teaching system based on differential evolution algorithmSoft Computing10.1007/s00500-023-08167-z27:13(9065-9075)Online publication date: 20-Apr-2023
  • (2019)Research and Development of a Zoom Synthesis Technology of Sound Field for Realizing Interactive Viewingインタラクティブ視聴を実現する音場のズーム合成技術の研究開発The Journal of The Institute of Image Information and Television Engineers10.3169/itej.73.16773:1(167-172)Online publication date: 2019
  • (2019)OtonoVR: Arbitrarily Angled Audio-visual VR Experience Using Selective Synthesis Sound Field TechniqueProceedings of the 27th ACM International Conference on Multimedia10.1145/3343031.3350602(2211-2213)Online publication date: 15-Oct-2019
  • (2018)Personal Viewpoint Navigation Based on Object Trajectory Distribution for Multi-View VideosIEICE Transactions on Information and Systems10.1587/transinf.2017EDP7122E101.D:1(193-204)Online publication date: 2018
  • (2016)F.Live: Towards interactive live broadcast FTV experienceIEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications10.1109/INFOCOM.2016.7524618(1-9)Online publication date: Apr-2016
  • (2016)Incentivizing crowdsourcing systems with network effectsIEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications10.1109/INFOCOM.2016.7524546(1-9)Online publication date: Apr-2016
  • (2013)[Invited paper] Latest Cable TV-Related Technologies and Services and Their Future ObservationITE Transactions on Media Technology and Applications10.3169/mta.1.531:1(53-59)Online publication date: 2013

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media