Ruizhao Chen
Zhigang Deng
ABSTRACT
We present "Double Doodles'' to make full use of two sequential inputs of a VR controller with 9 DOFs in total, 3 DOFs of the first input sequence for the generation of motion paths and 6 DOFs of the second input sequence for motion gestures. While engineering our system, we take ergonomics into consideration and design a set of user-defined motion gestures to describe character motions. We employ a real-time deep learning-based approach for highly accurate motion gesture classification. We then integrate our approach into a prototype system, and it allows users to directly create character animations in VR environments using motion gestures with a VR controller, followed by animation preview and animation interactive editing. Finally, we evaluate the feasibility and effectiveness of our system through a user study, demonstrating the usefulness of our system for visual storytelling dedicated to amateurs, as well as for providing fast drafting tools for artists.
- Adobe. 2022. Mixamo. https://www.mixamo.com.Google Scholar
- Raphael Anderegg, Loïc Ciccone, and Robert W. Sumner. 2018. PuppetPhone: Puppeteering Virtual Characters Using a Smartphone. In Proceedings of the 11th Annual International Conference on Motion, Interaction, and Games (Limassol, Cyprus) (MIG '18). Association for Computing Machinery, New York, NY, USA, Article 5, 6 pages. https://doi.org/10.1145/3274247.3274511Google ScholarDigital Library
- Aaron Bangor, Philip T. Kortum, and James T. Miller. 2008. An Empirical Evaluation of the System Usability Scale. Int. J. Hum. Comput. Interact., Vol. 24, 6 (2008), 574--594. https://doi.org/10.1080/10447310802205776Google ScholarCross Ref
- Amal Benzina, Marcus Toennis, Gudrun Klinker, and Mohamed Ashry. 2011. Phone-Based Motion Control in VR: Analysis of Degrees of Freedom. In CHI '11 Extended Abstracts on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI EA '11). Association for Computing Machinery, New York, NY, USA, 1519--1524. https://doi.org/10.1145/1979742.1979801Google ScholarDigital Library
- Ze Dong, Thammathip Piumsomboon, Jingjing Zhang, Adrian Clark, Huidong Bai, and Rob Lindeman. 2020. A Comparison of Surface and Motion User-Defined Gestures for Mobile Augmented Reality. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA '20). Association for Computing Machinery, New York, NY, USA, 1--8. https://doi.org/10.1145/3334480.3382883Google ScholarDigital Library
- Kraig Finstad. 2010. The Usability Metric for User Experience. Interacting with Computers, Vol. 22 (09 2010), 323--327. https://doi.org/10.1016/j.intcom.2010.04.004Google ScholarDigital Library
- Maxime Garcia, Remi Ronfard, and Marie-Paule Cani. 2019. Spatial Motion Doodles: Sketching Animation in VR Using Hand Gestures and Laban Motion Analysis. In Motion, Interaction and Games (Newcastle upon Tyne, United Kingdom) (MIG '19). Association for Computing Machinery, New York, NY, USA, Article 10, 10 pages. https://doi.org/10.1145/3359566.3360061Google ScholarDigital Library
- Google. 2022. Tilt Brush. https://www.tiltbrush.com/.Google Scholar
- Rebecca A. Grier. 2015. How High is High? A Meta-Analysis of NASA-TLX Global Workload Scores. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 59, 1 (2015), 1727--1731. https://doi.org/10.1177/1541931215591373Google ScholarCross Ref
- Jens Grubert, Eyal Ofek, Michel Pahud, and Per Ola Kristensson. 2018. The Office of the Future: Virtual, Portable, and Global. IEEE Comput. Graph. Appl., Vol. 38, 6 (Nov. 2018), 125--133. https://doi.org/10.1109/MCG.2018.2875609Google ScholarDigital Library
- Sandra G. Hart. 2006. Nasa-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 50, 9 (2006), 904--908. https://doi.org/10.1177/154193120605000909Google ScholarCross Ref
- Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015 (JMLR Workshop and Conference Proceedings, Vol. 37), Francis R. Bach and David M. Blei (Eds.). JMLR.org, 448--456. http://proceedings.mlr.press/v37/ioffe15.htmlGoogle Scholar
- Fabrizio Lamberti, Gianluca Paravati, Valentina Gatteschi, Alberto Cannavò, and Paolo Montuschi. 2018. Virtual Character Animation Based on Affordable Motion Capture and Reconfigurable Tangible Interfaces. IEEE Transactions on Visualization and Computer Graphics, Vol. 24, 5 (2018), 1742--1755. https://doi.org/10.1109/TVCG.2017.2690433Google ScholarCross Ref
- James R. Lewis and Jeff Sauro. 2009. The Factor Structure of the System Usability Scale. In Human Centered Design, First International Conference, HCD 2009, Held as Part of HCI International 2009, San Diego, CA, USA, July 19-24, 2009, Proceedings (Lecture Notes in Computer Science, Vol. 5619), Masaaki Kurosu (Ed.). Springer, 94--103. https://doi.org/10.1007/978-3-642-02806-9_12Google ScholarDigital Library
- Hui Liang, Jian Chang, Ismail Khalid Kazmi, Jian J. Zhang, and Peifeng Jiao. 2017. Hand gesture-based interactive puppetry system to assist storytelling for children. Vis. Comput., Vol. 33, 4 (2017), 517--531. https://doi.org/10.1007/s00371-016-1272-6Google ScholarDigital Library
- Aline Menin, Rafael Torchelsen, and Luciana Nedel. 2018. An Analysis of VR Technology Used in Immersive Simulations with a Serious Game Perspective. IEEE Comput. Graph. Appl., Vol. 38, 2 (March 2018), 57--73. https://doi.org/10.1109/MCG.2018.021951633Google ScholarDigital Library
- Juliana Montes and Pablo Figueroa. 2019. VR Salsa: Learning to Dance in Virtual Reality. In Proceedings of the IX Latin American Conference on Human Computer Interaction (Panama City, Panama) (CLIHC '19). Association for Computing Machinery, New York, NY, USA, Article 2, 4 pages. https://doi.org/10.1145/3358961.3358969Google ScholarDigital Library
- Francisco Javier Ordó n ez Morales and Daniel Roggen. 2016. Deep Convolutional and LS™ Recurrent Neural Networks for Multimodal Wearable Activity Recognition. Sensors, Vol. 16, 1 (2016), 115. https://doi.org/10.3390/s16010115Google ScholarCross Ref
- Ye Pan and Kenny Mitchell. 2020a. Group-Based Expert Walkthroughs: How Immersive Technologies Can Facilitate the Collaborative Authoring of Character Animation. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 188--195. https://doi.org/10.1109/VRW50115.2020.00041Google ScholarCross Ref
- Ye Pan and Kenny Mitchell. 2020b. PoseMMR: A Collaborative Mixed Reality Authoring Tool for Character Animation. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). 758--759. https://doi.org/10.1109/VRW50115.2020.00230Google ScholarCross Ref
- D.J. Sturman. 1998. Computer puppetry. IEEE Computer Graphics and Applications, Vol. 18, 1 (1998), 38--45. https://doi.org/10.1109/38.637269Google ScholarDigital Library
- Matthew Thorne, David Burke, and Michiel van de Panne. 2004. Motion Doodles: An Interface for Sketching Character Motion. ACM Trans. Graph., Vol. 23, 3 (Aug. 2004), 424--431. https://doi.org/10.1145/1015706.1015740Google ScholarDigital Library
- Thomas Tullis and William Albert. 2013. Measuring the User Experience, Second Edition: Collecting, Analyzing, and Presenting Usability Metrics 2nd ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.Google ScholarDigital Library
- Tvori. 2022. Tvori. https://tvori.co/.Google Scholar
- Andreia Valente, Augusto Esteves, and Daniel Lopes. 2021. From A-Pose to AR-Pose: Animating Characters in Mobile AR. In ACM SIGGRAPH 2021 Appy Hour (Virtual Event, USA) (SIGGRAPH '21). Association for Computing Machinery, New York, NY, USA, Article 4, 2 pages. https://doi.org/10.1145/3450415.3464401Google ScholarDigital Library
- Daniel Vogel, Paul Lubos, and Frank Steinicke. 2018. AnimationVR - Interactive Controller-based Animating in Virtual Reality. In 2018 IEEE 1st Workshop on Animation in Virtual and Augmented Environments (ANIVAE). 1--6. https://doi.org/10.1109/ANIVAE.2018.8587268Google ScholarCross Ref
- Hui Ye, Kin Chung Kwan, Wanchao Su, and Hongbo Fu. 2020. ARAnimator: In-Situ Character Animation in Mobile AR with User-Defined Motion Gestures. ACM Trans. Graph., Vol. 39, 4, Article 83 (July 2020), 12 pages. https://doi.org/10.1145/3386569.3392404Google ScholarDigital Library
- Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li. 2019. On the Continuity of Rotation Representations in Neural Networks. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5738--5746. https://doi.org/10.1109/CVPR.2019.00589Google ScholarCross Ref
Index Terms
- Double Doodles: Sketching Animation in Immersive Environment With 3+6 DOFs Motion Gestures
Recommendations
ARAnimator: in-situ character animation in mobile AR with user-defined motion gestures
Creating animated virtual AR characters closely interacting with real environments is interesting but difficult. Existing systems adopt video see-through approaches to indirectly control a virtual character in mobile AR, making close interaction with ...
Motion doodles: an interface for sketching character motion
In this paper we present a novel system for sketching the motion of a character. The process begins by sketching a character to be animated. An animated motion is then created for the character by drawing a continuous sequence of lines, arcs, and loops. ...
Spatial Motion Doodles: Sketching Animation in VR Using Hand Gestures and Laban Motion Analysis
MIG '19: Proceedings of the 12th ACM SIGGRAPH Conference on Motion, Interaction and GamesWe present a method for easily drafting expressive character animation by playing with instrumented rigid objects. We parse the input 6D trajectories (position and orientation over time) – called spatial motion doodles – into sequences of actions and ...
Comments