Abstract
Head-mounted displays (HMDs) are making virtual environments increasingly viable and real. As of the year 2021, some of the latest HMDs manufactured have incorporated cameras and/or sensors for the recognition and tracking of hands and facial expressions. These new devices include HTC-Vive-Focus-3 manufactured by HTC, HP-Reverb-G2-Omnicept-Edition manufactured by HP, Meta-Quest-Pro, manufactured by Meta, and Pico-4-Pro manufactured by Pico. A human's facial expressions convey emotional and non-verbal information. Transferring these expressions to build more realistic designs is a long-standing problem in computer animation. Recently, the development of facial reconstructions (2D and 3D) has achieved high performance, adjusting to being treatable in real time. There are different types of models for design and animation, more human and realistic models, unrealistic cartoon character models, and non-human models with different facial structures. Regardless of the design, there must be guarantees of a smooth transition between expressions so that the facial animation does not look choppy. This work aims to carry out a study of the main models for the design and animation of objects, which can reflect and support various types of human facial expressions obtained from the complete facial data provided by HMDs that incorporate cameras and/or sensors for face recognition and tracking.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
McLeod, R.: Animation Handbook (2019)
Mori, M., et al.: The Uncanny Valley [From the Field]. IEEE Robot. Autom. Mag. 19(2), 98–100 (2012). https://doi.org/10.1109/MRA.2012.2192811
Lewis, J., et al.: Practice and theory of Blendshape facial models. In: Eurographics 2014 - State of the Art Reports, p. 20 (2014). https://doi.org/10.2312/EGST.20141042
Shakir, S., Al-Azza, A.: Facial modelling and animation: an overview of the state-of-the Art. Iraqi J. Electr. Electron. Eng. 18(1), 28–37 (2022). https://doi.org/10.37917/ijeee.18.1.4
Noh, J.: A survey of facial modeling and animation techniques (2001). https://www.semanticscholar.org/paper/A-Survey-of-Facial-Modeling-and-Animation-Noh/9f1bbb74eb9f808421f46b924d8576bd46eb578a. Accessed 09 Mar 2023
Lee, Y., et al.: Realistic modeling for facial animation. In: Proceedings of the ACM SIGGRAPH Conference on Computer Graphics, pp. 55–62 (1995). https://doi.org/10.1145/218380.218407
Goodfellow, I., et al.: Generative Adversarial Networks, 10 June 2014. arXiv: https://doi.org/10.48550/arXiv.1406.2661.
Bansal, A., et al.: Recycle-GAN: unsupervised video retargeting, 15 August 2018. arXiv: https://doi.org/10.48550/arXiv.1808.05174
Montesinos López, O.A., Montesinos López, A., Crossa, J.: Convolutional Neural Networks. In: Montesinos López, O.A., Montesinos López, A., Crossa, J. (eds.) Multivariate Statistical Machine Learning Methods for Genomic Prediction, pp. 533–57710. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-89010-0_13
Deng, Y., et al.: Accurate 3D face reconstruction with weakly-supervised learning: from single image to image set, 09 April 2020. arXiv: [Online]. Available:http://arxiv.org/abs/1903.08527. Accessed: Mar. 09, 2023.
Lipton, Z., et al.: A critical review of recurrent neural networks for sequence learning, 17 October 2015. arXiv: http://arxiv.org/abs/1506.00019. Accessed 09 Mar 2023
Berson, E., et al.: Intuitive facial animation editing based on a generative RNN Framework, 12 October 2020. arXiv: http://arxiv.org/abs/2010.05655. Accessed 09 Mar 2023
Kingma, D., Welling, M.: An introduction to variational autoencoders. Found. Trends® Mach. Learn. 12(4), 307–392 (2019). https://doi.org/10.1561/2200000056
Lombardi, S., et al.: Deep appearance models for face rendering. ACM Trans. Graph. 37(4), 1–13 (2018). https://doi.org/10.1145/3197517.3201401
Egger, B., et al.: 3D Morphable face models -- past, present and future, 16 April 2020. arXiv: http://arxiv.org/abs/1909.01815. Accessed 14 Mar 2023
Jourabloo, A., et al.: Robust egocentric photo-realistic facial expression transfer for virtual reality. In: Proceedings IEEE Computer Society Conference Computer Vision Pattern Recognition, June 2022, pp. 20291–20300 (2022). https://doi.org/10.1109/CVPR52688.2022.01968
“Electromyography (EMG) - Mayo Clinic.” https://www.mayoclinic.org/tests-procedures/emg/about/pac-20393913. Accessed 14 Mar 2023
Lou, J., et al.: Realistic facial expression reconstruction for VR HMD users. IEEE Trans. Multimed. 22(3), 730–743 (2020). https://doi.org/10.1109/TMM.2019.2933338
Zhang, S., et al.: Automatic 3D face recovery from a single frame of a RGB-D sensor. In: 28th British Machine Vision Conference AFAHBU Workshop,” BMVA, August 2017. https://bmvc2017.london/. Accessed 13 Mar 2023
Yu, H., et al.: Perception-driven facial expression synthesis. Comput. Graph. 36(3), 152–162 (2012). https://doi.org/10.1016/j.cag.2011.12.002
Ozuysal, M., et al.: Fast keypoint recognition in ten lines of code, June 2007. https://doi.org/10.1109/CVPR.2007.383123
Acknowledgments
This paper was presented as part of the results of the Project “SIDIA-M_AR_Internet_For_Bondi”, carried out by the Institute of Science and Technology - SIDIA, in partnership with Samsung Eletrônica da Amazônia LTDA, in accordance with the Information Technology Law n.8387/ 91 and article at the. 39 of Decree 10,521/2020.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Nunes De Oliveira, D., Ortiz Díaz, A.A., Cleger Tamayo, S. (2023). Study of Different Methods to Design and Animate Realistic Objects for Virtual Environments on Modern HMDs. In: Stephanidis, C., Antona, M., Ntoa, S., Salvendy, G. (eds) HCI International 2023 Posters. HCII 2023. Communications in Computer and Information Science, vol 1836. Springer, Cham. https://doi.org/10.1007/978-3-031-36004-6_37
Download citation
DOI: https://doi.org/10.1007/978-3-031-36004-6_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36003-9
Online ISBN: 978-3-031-36004-6
eBook Packages: Computer ScienceComputer Science (R0)