Intelligent Film Assistant for Personalized Video Creation on Mobile Devices
Authors: 
Dominik Schörkhuber, Florian Seitner, Benedikt Salzbrunn, Margrit Gelautz, Georg BraunAuthors Info & Claims
Dominik Schörkhuber, Florian Seitner, Benedikt Salzbrunn, Margrit Gelautz, Georg BraunAuthors Info & ClaimsPages 210 - 215
Abstract
We describe the development of an intelligent film assistant system that supports the creation of professional video content on mobile devices such as smart phones by amateur users. Cinematographic expert knowledge on scene composition and camera motion is provided to the user in the form of story boards that are tailored to specific use cases. We give an overview of the project concept and some selected components including algorithms that are required for video stabilization and shot classification. The goal is to translate the scene characteristics detected by the vision algorithms into real-time feedback to the user during video recording and to support post-production. A major focus of the project is the incorporation of human-centered design principles with usability studies and expert interviews into the whole research process.
References
[1]
Subhabrata Bhattacharya, Ramin Mehran, Rahul Sukthankar, and Mubarak Shah. 2014. Classification of cinematographic shots using Lie algebra and its application to complex event recognition. IEEE Transactions on Multimedia 16, 3 (2014), 686--696.
[2]
Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2017. Realtime multi-person 2d pose estimation using part affinity fields. In 2017 IEEE Conference on Computer Vision and Pattern Recognition. http://openaccess.thecvf.com/content_cvpr_2017/papers/Cao_Realtime_Multi-Person_2D_CVPR_2017_paper.pdf
[3]
Scott Carter, John Adcock, John Doherty, and Stacy Branham. 2010. NudgeCam: Toward targeted, higher quality media capture. In Proceedings of the 18th ACM International Conference on Multimedia (MM '10). ACM, New York, NY, USA, 615--618.
[4]
Michael Gleicher and Feng Liu. 2008. Re-cinematography: Improving the camerawork of casual video. ACM Trans. Multimedia Comput. Commun. Appl. 5, 1 (2008), 1--28.
[5]
Michael L. Gleicher and Feng Liu. 2007. Re-cinematography. In Proceedings of the 15th ACM International Conference on Multimedia. ACM Press, New York, New York, USA, 27.
[6]
Matthias Grundmann, Vivek Kwatra, and Irfan Essa. 2011. Auto-directed video stabilization with robust L1 optimal camera paths. In 2011 IEEE Conference on Computer Vision and Pattern Recognition. 225--232.
[7]
Muhammad Abul Hasan, Min Xu, Xiangjian He, and Changsheng Xu. 2014. CAMHID: Camera motion histogram descriptor and its application to cinematographic shot classification. IEEE Transactions on Circuits and Systems for Video Technology 24, 10 (2014), 1682--1695.
[8]
Apple Inc. 2017. Human Interface Guidelines. (2017). https://developer.apple.com/ios/human-interface-guidelines/overview/themes/ {Online; accessed 5-October-2017}.
[9]
Google Inc. 2017. User Interface Guidelines. (2017). https://developer.android.com/guide/practices/ui_guidelines/index.html {Online; accessed 5-October-2017}.
[10]
ISO 9241-210:2010 2010. Ergonomics of human-system interaction part 210: human-centred design for interactive systems. Standard. International Organization for Standardization, Geneva, CH.
[11]
Steve Krug. 2010. Rocket surgery made easy. The do-it yourself guide to finding and fixing usability problems. New Riders, Canada (2010).
[12]
Tsung Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft COCO: Common objects in context. In Computer Vision -- ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V. Springer International Publishing, 740--755.
[13]
Christophe Lino, Marc Christie, Roberto Ranon, and William Bares. 2011. The director's lens: an intelligent assistant for virtual cinematography. In Proceedings of the 19th ACM International Conference on Multimedia. ACM, 323--332.
[14]
Feng Liu, Michael Gleicher, Hailin Jin, and Aseem Agarwala. 2009. Content-preserving warps for 3D video stabilization. ACM Transactions on Graphics 28, 3 (2009), 1.
[15]
Feng Liu, Michael Gleicher, Jue Wang, Hailin Jin, and Aseem Agarwala. 2011. Subspace video stabilization. ACM Trans. Graph. 30, 1, Article 4 (2011), 10 pages.
[16]
Shuaicheng Liu, Ping Tan, Lu Yuan, Jian Sun, and Bing Zeng. 2016. Mesh-Flow: Minimum latency online video stabilization. In Computer Vision -- ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VI. Springer International Publishing, 800--815.
[17]
Shuaicheng Liu, Lu Yuan, Ping Tan, and Jian Sun. 2013. Bundled camera paths for video stabilization. ACM Transactions on Graphics 32, 4 (2013), 1.
[18]
Shuaicheng Liu, Lu Yuan, Ping Tan, and Jian Sun. 2014. SteadyFlow: Spatially smooth optical flow for video stabilization. In 2014 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 4209--4216.
[19]
Jennifer McGinn and Ana Ramírez Chang. 2013. RITE+Krug: A combination of usability test methods for agile design. Journal of Usability Studies 8, 3 (2013), 61--68.
[20]
Michael C Medlock, Dennis Wixon, and Mark Terrano. 2002. Using the RITE method to improve products: A definition and a case study. Usability Professionals Association (2002).
[21]
Hiroko Mitarai. 2012. Interaction model for emotive video production. International Journal of Information and Electronics Engineering 2, 5 (2012), 661--666.
[22]
Hiroko Mitarai and Atsuo Yoshitaka. 2011. Shooting assistance by recognizing user's camera manipulation for intelligible video production. In 2011 IEEE International Symposium on Multimedia. 157--164.
[23]
Carlos Morimoto, Rama Chellappa, and Steve Balakirsky. 1997. Fast image stabilization and mosaicking. In DARPA Image Understanding Workshop (IUW).
[24]
Jakob Nielsen. 1989. Usability engineering at a discount. In Proceedings of the third International Conference on Human-Computer Interaction on Designing and using Human-Computer Interfaces and Knowledge Based Systems (2nd Ed.). 394--401.
[25]
George Papandreou, Tyler Zhu, Nori Kanazawa, Alexander Toshev, Jonathan Tompson, Chris Bregler, and Kevin Murphy. 2017. Towards accurate multi-person pose estimation in the wild. In 2017 IEEE Conference on Computer Vision and Pattern Recognition. http://openaccess.thecvf.com/content_cvpr_2017/papers/Papandreou_Towards_Accurate_Multi-Person_CVPR_2017_paper.pdf
[26]
Stephan R. Richter, Vibhav Vineet, Stefan Roth, and Vladlen Koltun. 2016. Playing for data: Ground truth from computer games. In Computer Vision --ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II. Springer International Publishing, 102--118.
[27]
Amy Schade. 2013. Competitive usability evaluations: Learning from your competition. (2013). https://www.nngroup.com/articles/competitive-usability-evaluations/ {Online; accessed 5-October-2017}.
Index Terms
- Intelligent Film Assistant for Personalized Video Creation on Mobile Devices
Recommendations
Feature Use in Mobile Video Creation
INTERACT '09: Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part IToday's mobile phones are also video cameras. People are using these ubiquitous cameras to document everyday surroundings as well as create more artistic videos. This paper examines emergent mobile film making patterns by tracking video composition and ...
A collaborative sketch animation creation system on mobile devices
CSCW '12: Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work CompanionIn this paper, we introduced a collaborative sketch animation system on mobile devices, enabling 2-users to create animation collaboratively in real time. By simple operations, the two users can not only sketch animation collaboratively but also ...
Film comic reflecting camera-works
MMM'12: Proceedings of the 18th international conference on Advances in Multimedia ModelingWe propose a novel technique for automatically creating film comics reflecting the camera-works of an original movie. Camera-works are one of the most important effects contributing to the mise en scene of the movie. A skilled director can use the ...
Comments
Information & Contributors
Information
Published In
December 2017
246 pages
ISBN:9781450353007
DOI:10.1145/3151848
Copyright © 2017 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
In-Cooperation
- Johannes Kepler University, Linz, Austria
- @WAS: International Organization of Information Integration and Web-based Applications and Services
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 04 December 2017
Check for updates
Qualifiers
- Short-paper
- Research
- Refereed limited
Funding Sources
- Vienna Business Agency
Conference
MoMM2017
MoMM2017: The 15th International Conference on Advances in Mobile Computing & Multimedia
December 4 - 6, 2017
Salzburg, Austria
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 81Total Downloads
- Downloads (Last 12 months)9
- Downloads (Last 6 weeks)2
Reflects downloads up to 07 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in