skip to main content
10.1145/3394171.3416528acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
abstract

Semantic Storytelling Automation: A Context-Aware and Metadata-Driven Approach

Published: 12 October 2020 Publication History

Abstract

Multimedia content production is nowadays widespread due to technological advances, namely supported by smartphones and social media. Although the massive amount of media content brings new opportunities to the industry, it also obfuscates the relevance of marketing content, meant to maintain and lure new audiences. This leads to an emergent necessity of producing these kinds of contents as quickly and engagingly as possible. Creating these automatically would decrease both the production costs and time, particularly by using static media for the creation of short storytelling animated clips. We propose an innovative approach that uses context and content information to transform a still photo into an appealing context-aware video clip. Thus, our solution presents a contribution to the state-of-the-art in computer vision and multimedia technologies and assists content creators with a value-added service to automatically build rich contextualized multimedia stories from single photographs.

Supplementary Material

MP4 File (3394171.3416528.mp4)
The work presented in this video proposes an innovative approach that uses context and content information to automatically transform a still photo into an appealing context-aware video clip. This enables decreasing both the production costs and time, particularly by using static media for the creation of short storytelling animated clips. The solution advances the state-of-the-art in computer vision and multimedia technologies in assisting content creators with a value-added service to automatically build rich contextualized multimedia stories from single photographs. The work was developed focusing on three main scenarios: photojournalism, fashion, and festivals. Nonetheless, the solution can be adapted to other creative industries or application scopes. The developed framework combines automated tools for context data extraction, object recognition and creative manipulations, merged to create smart immersive experiences.

References

[1]
Magisto, "Magisto," [Online]. Available: https://www.magisto.com/. [Accessed 01 04 2020]
[2]
Animoto, "Animoto," [Online]. Available: https://animoto.com/. [Accessed 01 04 2020].
[3]
Flixel, "Flixel," [Online]. Available: https://flixel.com/. [Accessed 01 04 2020].
[4]
Google, Google Cloud: Vision AI. [Online]. Available: https://cloud.google.com/vision. [Accessed 18 03 2020].
[5]
Microsoft Azure. Microsoft Azure: Computer Vision. [Online]. Available: https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/. [Accessed 18 03 2020].
[6]
Clarifai. Clarifai. [Online]. Available: https://www.clarifai.com/. [Accessed 18 03 2020].
[7]
J. Wang, J. Fu, J. Tang, Z. Li and T. Mei. 2018. Show, reward and tell: Automatic generation of narrative paragraph from photo stream by adversarial training. In Thirty-Second AAAI Conference on Artificial Intelligence.
[8]
X. Pan, F. Tang, W. Dong, C. Ma, Y. Meng, F. Huang, T.-Y. Lee and C. Xu. 2019. Content-Based Visual Summarization for Image Collections. IEEE Trans. on Visualization and Computer Graphics.
[9]
A. Singh, L. Virmani and A. Subramanyam. 2019. Image Corpus Representative Summarization. In 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM). pp 21--29. https://doi.org/10.1109/BigMM.2019.00--46
[10]
Y. Li, M. Geng, F. Liu and D. Zhang. 2019. Visualization of photo album: selecting a representative photo of a specific event. In International Conference on Database Systems for Advanced Applications. Lecture Notes in Computer Science, vol 11448. pp 128--141. https://doi.org/10.1007/978--3-030--18590--9_9
[11]
Intel. Intel Context Sensing SDK. 2014. [Online]. Available: https://www.youtube.com/watch?v=DX9wP7ZhAOY. [Accessed 16 03 2020].
[12]
C. Szegedy, S. Ioffe, V. Vanhoucke and A. A. Alemi. 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI Conference on Artificial Intelligence. pp. 4278--4284.
[13]
K. He, X. Zhang, S. Ren and J. Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770--778.

Cited By

View all
  • (2022)Boosting color similarity decisions using the CIEDE2000_PF MetricSignal, Image and Video Processing10.1007/s11760-022-02147-w16:7(1877-1884)Online publication date: 8-Feb-2022
  • (2021)Latent Memory-augmented Graph Transformer for Visual StorytellingProceedings of the 29th ACM International Conference on Multimedia10.1145/3474085.3475236(4892-4901)Online publication date: 17-Oct-2021
  • (2021)Inferring Contextual Data from Real-World PhotographyIntelligent Systems Design and Applications10.1007/978-3-030-71187-0_78(853-862)Online publication date: 3-Jun-2021

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MM '20: Proceedings of the 28th ACM International Conference on Multimedia
October 2020
4889 pages
ISBN:9781450379885
DOI:10.1145/3394171
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 October 2020

Check for updates

Author Tags

  1. computer vision
  2. metadata
  3. storytelling
  4. video generation

Qualifiers

  • Abstract

Funding Sources

  • European Commission

Conference

MM '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)17
  • Downloads (Last 6 weeks)1
Reflects downloads up to 03 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Boosting color similarity decisions using the CIEDE2000_PF MetricSignal, Image and Video Processing10.1007/s11760-022-02147-w16:7(1877-1884)Online publication date: 8-Feb-2022
  • (2021)Latent Memory-augmented Graph Transformer for Visual StorytellingProceedings of the 29th ACM International Conference on Multimedia10.1145/3474085.3475236(4892-4901)Online publication date: 17-Oct-2021
  • (2021)Inferring Contextual Data from Real-World PhotographyIntelligent Systems Design and Applications10.1007/978-3-030-71187-0_78(853-862)Online publication date: 3-Jun-2021

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media