Loading [a11y]/accessibility-menu.js
MetaFi++: WiFi-Enabled Transformer-Based Human Pose Estimation for Metaverse Avatar Simulation | IEEE Journals & Magazine | IEEE Xplore

MetaFi++: WiFi-Enabled Transformer-Based Human Pose Estimation for Metaverse Avatar Simulation

Publisher: IEEE

Abstract:

In the metaverse, digital avatar plays an important role in representing human beings for various interaction with virtual objects and environments, which puts a high dem...View more

Abstract:

In the metaverse, digital avatar plays an important role in representing human beings for various interaction with virtual objects and environments, which puts a high demand on effective pose estimation. Though camera-based solutions yield remarkable performance, they encounter privacy issues and degraded performance caused by varying illumination, especially in the smart home. In this article, we propose a WiFi-based Internet of Things-enabled human pose estimation scheme for metaverse avatar simulation, namely, MetaFi++. Specifically, WPFormer is designed with a shared convolutional module and a Transformer block to map the channel state information of WiFi signals to human pose landmarks, effectively exploring spatial information of human pose through self-attention. It is enforced to learn the annotations from the accurate computer vision model, thus achieving cross-modal supervision. Due to the ubiquitous existence of WiFi and robustness to various illumination conditions, WiFi-based human poses are suitable to instruct the movement of digital avatars in the metaverse, promoting avatar applications in smart homes. The experiments are conducted in the real world, and the results show that the MetaFi++ achieves very high performance with a PCK@50 of 97.30%. Our codes are available in https://github.com/pridy999/metafi_pose_estimation .
Published in: IEEE Internet of Things Journal ( Volume: 10, Issue: 16, 15 August 2023)
Page(s): 14128 - 14136
Date of Publication: 29 March 2023

ISSN Information:

Publisher: IEEE

References

References is not available for this document.