Alpana Dubey
ABSTRACT
Metaverse has received a huge attention in recent times with several Big Techs having invested in this concept. Accenture defines the metaverse as “an evolution of the Internet that enables a user to move beyond ‘browsing’ to ‘inhabiting’ in a persistent, shared experience that spans the spectrum of our real world to the fully virtual and in between”. The evolution that Metaverse brings can be seen along three dimensions: 1) shift towards spatial experiences: which includes 2D, 3D, augmented, virtual, and mixed reality immersive experiences, 2) shared co-presence: where users experience a persistent shared space with a sense of co-presence with others, and 3) trusted identities and transactions to address challenges of fake identities, products, and transactions as present in today’s internet.
For example, a retail marketplace, on Metaverse could be seen as an immersive spatial experience where users can shop along with their families and friends who join virtually in the same environment. The sense of shared co-presence gives them the ability to discuss about products in real time and persistency gives them ability to come back to the same space. This evolution opens an enormous opportunity to rethink the digital experiences future applications would offer to the people. AI would be the core engine behind making these experiences richer, immersive, and engaging. The role of AI, in the Metaverse, is broad; however, in this tutorial, we will focus on two areas where AI will play a major role in shaping up the form and function of the Metaverse by: 1) bringing more realism in Metaverse with high fidelity immersive content generated through AI techniques and 2) enhancing user interactions by bringing more intelligence in the interaction modes.
- 2022. AvatarSDK. (2022). https://avatarsdk.com [Online; accessed 19-Nov-2022].Google Scholar
- 2022. DeepMotion. (2022). https://www.deepmotion.com [Online; accessed 19-Nov-2022].Google Scholar
- 2022. Expert, Natural Q&A with NVIDIA Omniverse Avatar for Project Tokkio. (2022). https://www.youtube.com/watch?v=U9Zh57dGsH4 [Online; accessed 19-Nov-2022].Google Scholar
- 2022. GANverse3D: a neural network from NVIDIA reconstructs a 3D Model from a single photo. (2022). https://neurohive.io/en/news/ganverse3d-a-neural-network-from-nvidia-reconstructs-a-3d-model-from-a-single-photo/ [Online; accessed 19-Nov-2022].Google Scholar
- 2022. Into The Metaverse: The Future Of Virtual Interactions. (2022). https://www.forbes.com/sites/forbestechcouncil/2022/07/11/into-the-metaverse-the-future-of-virtual-interactions/ [Online; accessed 19-Nov-2022].Google Scholar
- 2022. Omniverse Audio2Face: Generate expressive facial animation from just an audio source with NVIDIA’s Deep Learning AI technology. (2022). https://www.nvidia.com/en-in/omniverse/apps/audio2face/ [Online; accessed 19-Nov-2022].Google Scholar
- 2022. Project CAIRaoke: Building the assistants of the future with breakthroughs in conversational AI. (2022). https://ai.facebook.com/blog/project-cairaoke/ [Online; accessed 19-Nov-2022].Google Scholar
- 2022. Project Starline: Feel like you’re there, together. (2022). https://www.youtube.com/watch?v=Q13CishCKXY [Online; accessed 19-Nov-2022].Google Scholar
- 2022. Teaching AI to translate 100s of spoken and written languages in real time. (2022). https://ai.facebook.com/blog/teaching-ai-to-translate-100s-of-spoken-and-written-languages-in-real-time/ [Online; accessed 19-Nov-2022].Google Scholar
- 2022. UnrealEngine’s MetaHuman. (2022). https://www.unrealengine.com/en-US/metahuman [Online; accessed 19-Nov-2022].Google Scholar
- Louis Airale, Dominique Vaufreydaz, and Xavier Alameda-Pineda. 2022. Socialinteractiongan: Multi-person interaction sequence generation. IEEE Transactions on Affective Computing(2022).Google Scholar
- Nitish Bhardwaj, Dhornala Bharadwaj, and Alpana Dubey. 2022. SingleSketch2Mesh: Generating 3D Mesh model from Sketch. arXiv preprint arXiv:2203.03157(2022).Google Scholar
- Zehranaz Canfes, M Furkan Atasoy, Alara Dirik, and Pinar Yanardag. 2022. Text and Image Guided 3D Avatar Generation and Manipulation. arXiv preprint arXiv:2202.06079(2022).Google Scholar
- Aysegul Dundar, Jun Gao, Andrew Tao, and Bryan Catanzaro. 2022. Fine Detailed Texture Learning for 3D Meshes with Generative Models. arXiv preprint arXiv:2203.09362(2022).Google Scholar
- Yao Feng, Haiwen Feng, Michael J Black, and Timo Bolkart. 2021. Learning an animatable detailed 3D face model from in-the-wild images. ACM Transactions on Graphics (ToG) 40, 4 (2021), 1–13.Google ScholarDigital Library
- Thien Huynh-The, Quoc-Viet Pham, Xuan-Qui Pham, Thanh Thi Nguyen, Zhu Han, and Dong-Seong Kim. 2022. Artificial Intelligence for the Metaverse: A Survey. arXiv preprint arXiv:2202.10336(2022).Google Scholar
- Ting-En Lin, Yuchuan Wu, Fei Huang, Luo Si, Jian Sun, and Yongbin Li. 2022. Duplex Conversation: Towards Human-like Interaction in Spoken Dialogue System. arXiv preprint arXiv:2205.15060(2022).Google Scholar
- Zhaoliang Lun, Matheus Gadelha, Evangelos Kalogerakis, Subhransu Maji, and Rui Wang. 2017. 3D shape reconstruction from sketches via multi-view convolutional networks. In 2017 International Conference on 3D Vision (3DV). IEEE, 67–77.Google ScholarCross Ref
- Oscar Michel, Roi Bar-On, Richard Liu, Sagie Benaim, and Rana Hanocka. 2022. Text2mesh: Text-driven neural stylization for meshes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13492–13502.Google ScholarCross Ref
- Charlie Nash, Yaroslav Ganin, SM Ali Eslami, and Peter Battaglia. 2020. Polygen: An autoregressive generative model of 3d meshes. In International conference on machine learning. PMLR, 7220–7229.Google Scholar
- Sergey Prokudin, Michael J Black, and Javier Romero. 2021. SMPLpix: Neural Avatars from 3D Human Models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 1810–1819.Google ScholarCross Ref
- Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M Smith, 2020. Recipes for building an open-domain chatbot. arXiv preprint arXiv:2004.13637(2020).Google Scholar
- Shunsuke Saito, Tomas Simon, Jason Saragih, and Hanbyul Joo. 2020. Pifuhd: Multi-level pixel-aligned implicit function for high-resolution 3d human digitization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 84–93.Google ScholarCross Ref
- Abhinav Upadhyay, Alpana Dubey, Suma Mani Kuriakose, and Devasish Mahato. 2022. 3DSTNet: Neural 3D Shape Style Transfer. In 2022 IEEE International Conference on Multimedia and Expo Workshops (ICMEW). IEEE, 1–6.Google ScholarCross Ref
- Kangxue Yin, Jun Gao, Maria Shugrina, Sameh Khamis, and Sanja Fidler. 2021. 3dstylenet: Creating 3d shapes with geometric and texture style variations. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 12456–12465.Google ScholarCross Ref
- Song-Hai Zhang, Yuan-Chen Guo, and Qing-Wen Gu. 2021. Sketch2Model: View-aware 3d modeling from single free-hand sketches. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6012–6021.Google ScholarCross Ref
Index Terms
- AI for Immersive Metaverse Experience
Recommendations
Virtual Reality, Augmented Reality and Metaverse: Customer Experience Approach and User Experience Evaluation Methods. Literature Review
Social Computing and Social MediaAbstractCurrently, the technologies associated with the web and telecommunications have an essential role in different areas, such as education, medicine, tourism, commerce, among others. The coronavirus pandemic (COVID-19) accelerated the process of ...
Virtual reality consumer experience escapes: preparing for the metaverse
AbstractVirtual Reality (VR) experience escapes allow individuals to spend hours on end in immersive virtual environments and interact with content in a world that is providing shelter and illusion of an alternative reality – the metaverse. Discussions on ...
Ukemochi: A Video See-through Food Overlay System for Eating Experience in the Metaverse
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing SystemsThe widespread use of Head-Mounted Displays (HMDs) allows ordinary users to interact with their friends daily in social Virtual Environments (VEs) or metaverse. However, it is not easy to eat in a metaverse while wearing an HMD because the Real ...
Comments