Erwin Wu
Takashi Matsumoto
Hideki Koike
ABSTRACT
Effective analysis of skills requires high-quality, multi-modal datasets, especially in the field of artificial intelligence. However, creating such datasets for extreme sports, such as alpine skiing, can be challenging due to environmental constraints. Optical and wearable sensors may not perform optimally under diverse lighting, weather, and terrain conditions. To address these challenges, we present a comprehensive skiing/snowboarding dataset using a professional motor-based simulator. Using the realistic simulator, it is easy to obtain different types of data with a small domain gap between real-world data. Common data for skill analysis are collected, including camera images, 3D body pose, sole pressure, and leg electromyography, from athletes of different levels. Another key aspect is the comparison of cross-modal baselines, highlighting the versatility of the data across modalities. In addition, a real-world pilot test is conducted to assess the practical applicability and data robustness.
Supplemental Material
- Leandro Abraham, Facundo Bromberg, and Raymundo Forradellas. 2015. Arm muscular effort estimation from images using computer vision and machine learning. In Ambient Intelligence for Health. Springer, 125--137.Google Scholar
- Hyemin Ahn, Youssef Michel, Thomas Eiband, and Dongheui Lee. 2023. Vision-Based Approximate Estimation of Muscle Activation Patterns for Tele-Impedance. IEEE Robotics and Automation Letters , Vol. 8, 8 (2023), 5220--5227. https://doi.org/10.1109/LRA.2023.3293275Google ScholarCross Ref
- Roman Bachmann, Jörg Spörri, Pascal Fua, and Helge Rhodin. 2019. Motion capture from pan-tilt cameras with unknown orientation. In 2019 International Conference on 3D Vision (3DV). IEEE, 308--317.Google ScholarCross Ref
- bronkscottema. 2023. Football Players Dataset. https://universe.roboflow.com/bronkscottema/football-players-zm06l. https://universe.roboflow.com/bronkscottema/football-players-zm06l visited on 2023-07--21.Google Scholar
- Leslie Casas, Chris Mürwald, Felix Achilles, Diana Mateus, Dietrich Huber, Nassir Navab, and Stefanie Demirci. 2018. Human pose estimation from pressure sensor data. In Bildverarbeitung für die Medizin 2018: Algorithmen-Systeme-Anwendungen. Proceedings des Workshops vom 11. bis 13. M"arz 2018 in Erlangen. Springer, 285--290.Google Scholar
- Leslie Casas, Nassir Navab, and Stefanie Demirci. 2019. Patient 3D body pose estimation from pressure imaging. International journal of computer assisted radiology and surgery , Vol. 14 (2019), 517--524.Google ScholarCross Ref
- Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).Google Scholar
- Yixuan Li, Lei Chen, Runyu He, Zhenzhi Wang, Gangshan Wu, and Limin Wang. 2021. Multisports: A multi-person video dataset of spatio-temporally localized sports actions. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 13536--13545.Google ScholarCross Ref
- Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6--12, 2014, Proceedings, Part V 13. Springer, 740--755.Google Scholar
- Yilin Liu, Shijia Zhang, and Mahanth Gowda. 2021. NeuroPose: 3D hand pose tracking using EMG wearables. In Proceedings of the Web Conference 2021. 1471--1482.Google ScholarDigital Library
- Katja Ludwig, Moritz Einfalt, and Rainer Lienhart. 2020. Robust Estimation of Flight Parameters for SKI Jumpers. In 2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). 1--6. https://doi.org/10.1109/ICMEW46912.2020.9105973Google ScholarCross Ref
- Julieta Martinez, Rayat Hossain, Javier Romero, and James J Little. 2017. A simple yet effective baseline for 3d human pose estimation. In Proceedings of the IEEE international conference on computer vision. 2640--2649.Google ScholarCross Ref
- Dushyant Mehta, Oleksandr Sotnychenko, Franziska Mueller, Weipeng Xu, Mohamed Elgharib, Pascal Fua, Hans-Peter Seidel, Helge Rhodin, Gerard Pons-Moll, and Christian Theobalt. 2020. XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera. ACM Transactions on Graphics , Vol. 39, 4, bibinfonumpages17 pages. https://doi.org/10.1145/3386569.3392410Google ScholarDigital Library
- Dushyant Mehta, Srinath Sridhar, Oleksandr Sotnychenko, Helge Rhodin, Mohammad Shafiei, Hans-Peter Seidel, Weipeng Xu, Dan Casas, and Christian Theobalt. 2017. VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera. ACM Transactions on Graphics , Vol. 36, 4, bibinfonumpages14 pages. https://doi.org/10.1145/3072959.3073596Google ScholarDigital Library
- MM4SPA. [n.,d.]. Multimodal Analysis for Sports Analytics. https://mm4spa.github.io/project/Google Scholar
- Francesc Moreno-Noguer. 2017. 3d human pose estimation from a single image via distance matrix regression. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2823--2832.Google ScholarCross Ref
- Kunyu Peng, David Schneider, Alina Roitberg, Kailun Yang, Jiaming Zhang, M Saquib Sarfraz, and Rainer Stiefelhagen. 2023. MuscleMap: Towards Video-based Activated Muscle Group Estimation. arXiv preprint arXiv:2303.00952 (2023).Google Scholar
- Jesse Scott, Bharadwaj Ravichandran, Christopher Funk, Robert T. Collins, and Yanxi Liu. 2020. From Image to Stability: Learning Dynamics from Human Pose. In Computer Vision -- ECCV 2020, , Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.). Springer International Publishing, Cham, 536--554.Google ScholarDigital Library
- Jörg Spörri. 2016. Reasearch dedicated to sports injury prevention-the'sequence of prevention'on the example of alpine ski racing. Habilitation with Venia Docendi in Biomechanics, Vol. 1, 2 (2016), 7.Google Scholar
- Gabriel Van Zandycke and Christophe De Vleeschouwer. 2019. Real-time CNN-based segmentation architecture for ball detection in a single view setup. In Proceedings Proceedings of the 2nd International Workshop on Multimedia Content Analysis in Sports. 51--58.Google ScholarDigital Library
- Roman Voeikov, Nikolay Falaleev, and Ruslan Baikulov. 2020. TTNet: Real-Time Temporal and Spatial Video Analysis of Table Tennis. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.Google Scholar
- Xi Wei, Tianzhu Zhang, Yan Li, Yongdong Zhang, and Feng Wu. 2020. Multi-modality cross attention network for image and sentence matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10941--10950.Google ScholarCross Ref
- Erwin Wu, Takayuki Nozawa, Florian Perteneder, and Hideki Koike. 2020. VR Alpine Ski Training Augmentation Using Visual Cues of Leading Skier. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.Google ScholarCross Ref
- Erwin Wu, Florian Perteneder, Hideki Koike, and Takayuki Nozawa. 2019. How to vizski: Visualizing captured skier motion in a vr ski training simulator. In Proceedings of the 17th International Conference on Virtual-Reality Continuum and its Applications in Industry. 1--9.Google ScholarDigital Library
- KangKang Yin and Dinesh K Pai. 2003. FootSee: an interactive animation system.. In Symposium on Computer Animation. 329--338.Google Scholar
- Masairo Yoshikawa, Masahiko Mikawa, and Kazuyo Tanaka. 2007. Hand pose estimation using EMG signals. In 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, 4830--4833.Google ScholarCross Ref
- Qitao Zhao, Ce Zheng, Mengyuan Liu, Pichao Wang, and Chen Chen. 2023. PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimation. arXiv preprint arXiv:2303.17472 (2023).Google Scholar
- Ce Zheng, Sijie Zhu, Matias Mendieta, Taojiannan Yang, Chen Chen, and Zhengming Ding. 2021. 3d human pose estimation with spatial and temporal transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 11656--11665.Google ScholarCross Ref
Index Terms
- SkiTech: An Alpine Skiing and Snowboarding Dataset of 3D Body Pose, Sole Pressure, and Electromyography
Recommendations
Development of a whole-body walking rehabilitation robot and power assistive method using EMG signals
AbstractIn this paper, we describe the development of DDGo Advanced, a whole-body walking rehabilitation robot designed as an end-effector type, along with its three walking training methods. Most of the walking rehabilitation robots currently on the ...
Body sensor networks to evaluate standing balance: interpreting muscular activities based on inertial sensors
HealthNet '08: Proceedings of the 2nd International Workshop on Systems and Networking Support for Health Care and Assisted Living EnvironmentsIn this paper, we present a system that integrates inertial sensors and electromyogram (EMG) signals, which measures the muscular activities while performing motions. The objective of our study is to investigate the behaviour of the EMG signals to ...
Gait rehabilitation system for stair climbing and descending
HAPTIC '10: Proceedings of the 2010 IEEE Haptics SymposiumThis paper describes the development of a gait rehabilitation system with a locomotion interface (LI) for stair climbing or descending. The LI consists of two 2 DOF manipulators equipped with footpads. These can move the user's feet while his or her ...
Comments