AudioMove: Applying the Spatial Audio to Multi-Directional Limb Exercise Guidance
Article No.: 244, Pages 1 - 26
Abstract
Guiding users with limb exercise can assist in muscle training or physical recovery. However, traditional vision-based methods often require multiple camera angles to help users understand the motions and require them to be within the range of the screen. Therefore, we propose a non-visual system that can guide users with multiple-directional limb motions utilizing spatial audio, AudioMove, with commercial-off-the-shelf (COTS) devices (i.e., smartphones and earphones). The proposed system addresses the challenge of conveying directional information encompassing multiple planes in real-time. We conduct a mixed-method user study to evaluate the effectiveness of the system with three methods combining motion data with spatial audio perception. Additionally, a user interface is built to collect users' comments. The results conclude that spatial audio guidance could create a natural, pervasive, and non-visual exercise training solution in daily life.
Supplemental Material
MP4 File
Supplemental video
- Download
- 167.57 MB
References
[1]
Fraser Anderson, Tovi Grossman, Justin Matejka, and George Fitzmaurice. 2013. YouMove: enhancing movement training with an augmented reality mirror. In Proceedings of the 26th annual ACM symposium on User interface software and technology. ACM, New York, NY, USA, 311--320.
[2]
Apple. 2023. Airpods. https://www.apple.com/airpods/.
[3]
Riku Arakawa and Hiromu Yakura. 2021. Mindless Attractor: A False-Positive Resistant Intervention for Drawing Attention Using Auditory Perturbation. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1--15.
[4]
Riku Arakawa, Hiromu Yakura, Vimal Mollyn, Suzanne Nie, Emma Russell, Dustin P. DeMeo, Haarika A. Reddy, Alexander K. Maytin, Bryan T. Carroll, Jill Fain Lehman, and Mayank Goel. 2023. PrISM-Tracker: A Framework for Multimodal Procedure Tracking Using Wearable Sensors and State Transition Information with User-Driven Handling of Errors and Uncertainty. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 6, 4, Article 156 (jan 2023), 27 pages. https://doi.org/10.1145/3569504
[5]
Mobolaji Ayoade and Lynne Baillie. 2014. A novel knee rehabilitation system for the home. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, NY, USA, 2521--2530.
[6]
David G Behm, Anthony J Blazevich, Anthony D Kay, and Malachy McHugh. 2016. Acute effects of muscle stretching on physical performance, range of motion, and injury incidence in healthy active individuals: a systematic review. Applied physiology, nutrition, and metabolism 41, 1 (2016), 1--11.
[7]
Bose. 2023. Bose. https://www.bose.com/en_us/landing_pages/developer-portal.html.
[8]
Virginia Braun and Victoria Clarke. 2019. Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health 11, 4 (2019), 589--597. https://doi.org/10.1080/2159676X.2019.1628806 arXiv:https://doi.org/10.1080/2159676X.2019.1628806
[9]
James Broderick, Jim Duggan, and Sam Redfern. 2018. The importance of spatial audio in modern games and virtual environments. In 2018 IEEE Games, Entertainment, Media Conference (GEM). IEEE, 1--9.
[10]
Qinglei Bu, Xiaoyi Cheng, Fan Yang, Jie Sun, Limin Yu, and Ying Hou. 2022. A Computer Game-based Tangible Upper Limb Rehabilitation Device. In Proceedings of the 10th International Conference on Human-Agent Interaction. 309--313.
[11]
Yuanzhi Cao, Xun Qian, Tianyi Wang, Rachel Lee, Ke Huo, and Karthik Ramani. 2020. An Exploratory Study of Augmented Reality Presence for Tutoring Machine Tasks. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1--13.
[12]
Filippo Casamassima, Alberto Ferrari, Bojan Milosevic, Laura Rocchi, and Elisabetta Farella. 2013. Wearable audiofeedback system for gait rehabilitation in subjects with Parkinson?s disease. In Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication. ACM, New York, NY, USA, 275--278.
[13]
Yao-Jen Chang, Shu-Fang Chen, and Jun-Da Huang. 2011. A Kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities. Research in developmental disabilities 32, 6 (2011), 2566--2570.
[14]
Philo Tan Chua, Rebecca Crivella, Bo Daly, Ning Hu, Russ Schaaf, David Ventura, Todd Camill, Jessica Hodgins, and Randy Pausch. 2003. Training for physical tasks in virtual environments: Tai Chi. In IEEE Virtual Reality, 2003. Proceedings. IEEE, Long Beach, CA, USA, 87--94.
[15]
Christopher Clarke, Doga Cavdir, Patrick Chiu, Laurent Denoue, and Don Kimber. 2020. Reactive Video: Adaptive Video Playback Based on User Motion for Supporting Physical Activity. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. ACM, New York, NY, USA, 196--208.
[16]
Sebastian Cmentowski, Sukran Karaosmanoglu, Lennart E. Nacke, Frank Steinicke, and Jens Harald Krüger. 2023. Never Skip Leg Day Again: Training the Lower Body with Vertical Jumps in a Virtual Reality Exergame. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI '23). Association for Computing Machinery, New York, NY, USA, Article 786, 18 pages. https://doi.org/10.1145/3544548.3580973
[17]
Alexandra Covaci, Anne-Hélène Olivier, and Franck Multon. 2015. Visual perspective and feedback guidance for vr free-throw training. IEEE computer graphics and applications 35, 5 (2015), 55--65.
[18]
Alana Elza Fontes Da Gama, Thiago de Menezes Chaves, Pascal Fallavollita, Lucas Silva Figueiredo, and Veronica Teichrieb. 2019. Rehabilitation motion recognition based on the international biomechanical standards. Expert Systems with Applications 116 (2019), 396--409.
[19]
Christof Faller. 2004. Parametric coding of spatial audio. Technical Report. EPFL.
[20]
Jamie Ferguson, Lorna Paul, and Stephen Brewster. 2021. A Peripheral Tactile Feedback System for Lateral Epicondilytus Rehabilitation Exercise. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 1--6.
[21]
Antonio Frisoli, Caterina Procopio, Carmelo Chisari, Ilaria Creatini, Luca Bonfiglio, Massimo Bergamasco, Bruno Rossi, and Maria Chiara Carboncini. 2012. Positive effects of robotic exoskeleton training of upper limb reaching movements after stroke. Journal of neuroengineering and rehabilitation 9, 1 (2012), 1--16.
[22]
Nadia Garcia-Hernandez and Vicente Parra-Vega. 2009. Active and efficient motor skill learning method used in a haptic teleoperated system. In RO-MAN 2009-The 18th IEEE International Symposium on Robot and Human Interactive Communication. IEEE, 915--920.
[23]
Jeremy Raboff Gordon, Alexander J. Fiannaca, Melanie Kneisel, Edward Cutrell, Amos Miller, and Mar Gonzalez-Franco. 2023. Hearing the Way Forward: Exploring Ambient Navigational Awareness with Reduced Cognitive Load through Spatial Audio-AR. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI EA '23). Association for Computing Machinery, New York, NY, USA, Article 155, 7 pages. https://doi.org/10.1145/3544549.3585800
[24]
Natsuki Hamanishi and Jun Rekimoto. 2020. PoseAsQuery: Full-Body Interface for Repeated Observation of a Person in a Video with Ambiguous Pose Indexes and Performed Poses. In Proceedings of the Augmented Humans International Conference. ACM, New York, NY, USA, 1--11.
[25]
Ping-Hsuan Han, Yang-Sheng Chen, Yilun Zhong, Han-Lei Wang, and Yi-Ping Hung. 2017. My Tai-Chi coaches: an augmented-learning tool for practicing Tai-Chi Chuan. In Proceedings of the 8th Augmented Human International Conference. ACM, New York, NY, USA, 1--4.
[26]
Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139--183.
[27]
WILLIAM HARTMANN and Eric Macaulay. 2014. Anatomical limits on interaural time differences: an ecological perspective. Frontiers in Neuroscience 8 (2014). https://doi.org/10.3389/fnins.2014.00034
[28]
Thuong N Hoang, Martin Reinoso, Frank Vetere, and Egemen Tanin. 2016. Onebody: remote posture guidance system using first person view in virtual environment. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction. ACM, New York, NY, USA, 1--10.
[29]
Eve Hoggan, Andrew Crossan, Stephen A Brewster, and Topi Kaaresoja. 2009. Audio or tactile feedback: which modality when?. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, NY, USA, 2253--2256.
[30]
Xuhui Hu, Aiguo Song, Zhikai Wei, and Hong Zeng. 2022. StereoPilot: A wearable target location system for blind and visually impaired using spatial audio rendering. IEEE Transactions on Neural Systems and Rehabilitation Engineering 30 (2022), 1621--1630.
[31]
Xuhui Hu, Aiguo Song, Hong Zeng, and Dapeng Chen. 2022. Intuitive environmental perception assistance for blind amputees using spatial audio rendering. IEEE Transactions on Medical Robotics and Bionics 4, 1 (2022), 274--284.
[32]
Masahiko Inami, Daisuke Uriu, Zendai Kashino, Shigeo Yoshida, Hiroto Saito, Azumi Maekawa, and Michiteru Kitazaki. 2022. Cyborgs, Human Augmentation, Cybernetics, and JIZAI Body. In Proceedings of the Augmented Humans International Conference 2022 (Kashiwa, Chiba, Japan) (AHs '22). Association for Computing Machinery, New York, NY, USA, 230--242. https://doi.org/10.1145/3519391.3519401
[33]
Md Shafiqul Islam and Sol Lim. 2022. Vibrotactile feedback in virtual motor learning: A systematic review. Applied Ergonomics 101 (2022), 103694.
[34]
Norihide Itoh, Hitoshi Kagaya, Eiichi Saitoh, Kei Ohtsuka, Junya Yamada, Hiroki Tanikawa, Shigeo Tanabe, Naoki Itoh, Takemitsu Aoki, and Yoshikiyo Kanada. 2012. Quantitative assessment of circumduction, hip hiking, and forefoot contact gait using Lissajous figures. Japanese Journal of Comprehensive Rehabilitation Science 3 (2012), 78--84.
[35]
Florian Jeanne, Yann Soullard, Ali Oker, and Indira Thouvenin. 2017. EBAGG: Error-based assistance for gesture guidance in virtual environments. In 2017 IEEE 17th International Conference on Advanced Learning Technologies (ICALT). IEEE, Long Beach, CA, USA, 472--476.
[36]
Florian Jeanne, Indira Thouvenin, and Alban Lenglet. 2017. A study on improving performance in gesture training through visual guidance based on learners? errors. In Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology. ACM, New York, NY, USA, 1--10.
[37]
Daniel Johnston, Hauke Egermann, and Gavin Kearney. 2019. Measuring the behavioral response to spatial audio within a multi-modal virtual reality environment in children with autism spectrum disorder. Applied Sciences 9, 15 (2019), 3152.
[38]
Jakob Karolus, Felix Bachmann, Thomas Kosch, Albrecht Schmidt, and Pawel W. Wozniak. 2021. Facilitating Bodily Insights Using Electromyography-Based Biofeedback during Physical Activity. In Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction (Toulouse & Virtual, France) (MobileHCI '21). Association for Computing Machinery, New York, NY, USA, Article 14, 15 pages. https://doi.org/10.1145/3447526.3472027
[39]
Rushil Khurana, Karan Ahuja, Zac Yu, Jennifer Mankoff, Chris Harrison, and Mayank Goel. 2018. GymCam: Detecting, recognizing and tracking simultaneous exercises in unconstrained scenes. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 4 (2018), 1--17.
[40]
Yousef Kowsar, Masud Moshtaghi, Eduardo Velloso, Lars Kulik, and Christopher Leckie. 2016. Detecting unseen anomalies in weight training exercises. In Proceedings of the 28th Australian Conference on Computer-Human Interaction. ACM, New York, NY, USA, 517--526.
[41]
Kartikaeya Kumar, Lev Poretski, Jiannan Li, and Anthony Tang. 2022. Tourgether360: Collaborative Exploration of 360° Videos Using Pseudo-Spatial Navigation. Proc. ACM Hum.-Comput. Interact. 6, CSCW2, Article 546 (nov 2022), 27 pages. https://doi.org/10.1145/3555604
[42]
Sih-Pin Lai, Cheng-An Hsieh, Yu-Hsin Lin, Teepob Harutaipree, Shih-Chin Lin, Yi-Hao Peng, Lung-Pan Cheng, and Mike Y Chen. 2020. StrengthGaming: Enabling dynamic repetition tempo in strength training-based exergame design. In 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services. 1--8.
[43]
Marcel B Lanza, Thomas G Balshaw, and Jonathan P Folland. 2019. Is the joint-angle specificity of isometric resistance training real? And if so, does it have a neural basis? European Journal of Applied Physiology 119 (2019), 2465--2476.
[44]
Jiann-Der Lee, Chung-Hung Hsieh, and Ting-Yang Lin. 2014. A Kinect-based Tai Chi exercises evaluation system for physical rehabilitation. In 2014 IEEE International Conference on Consumer Electronics (ICCE). IEEE, Long Beach, CA, USA, 177--178.
[45]
Zhipeng Li, Yu Jiang, Yihao Zhu, Ruijia Chen, Ruolin Wang, Yuntao Wang, Yukang Yan, and Yuanchun Shi. 2022. Modeling the Noticeability of User-Avatar Movement Inconsistency for Sense of Body Ownership Intervention. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 2 (2022), 1--26.
[46]
Jeff Lieberman and Cynthia Breazeal. 2007. TIKL: Development of a wearable vibrotactile feedback suit for improved human motor learning. IEEE Transactions on Robotics 23, 5 (2007), 919--926.
[47]
James Makous and John Middlebrooks. 1990. Two-dimensional sound localization by human listeners. The Journal of the Acoustical Society of America 87 (06 1990), 2188--200. https://doi.org/10.1121/1.399186
[48]
Alexander Marquardt, Christina Trepkowski, Tom David Eibich, Jens Maiero, Ernst Kruijff, and Johannes Schöning. 2020. Comparing non-visual and visual guidance methods for narrow field of view augmented reality displays. IEEE Transactions on Visualization and Computer Graphics 26, 12 (2020), 3389--3401.
[49]
Troy L McDaniel, Morris Goldberg, Shantanu Bala, Bijan Fakhri, and Sethuraman Panchanathan. 2012. Vibrotactile feedback of motor performance errors for enhancing motor learning. In Proceedings of the 20th ACM international conference on Multimedia. ACM, New York, NY, USA, 419--428.
[50]
Catarina Mendonça and Victoria Korshunova. 2020. Surround sound spreads visual attention and increases cognitive effort in immersive media reproductions. In Proceedings of the 15th International Audio Mostly Conference. 16--21.
[51]
Microsoft. 2023. Microsoft Soundscape. https://www.microsoft.com/en-us/research/product/soundscape/.
[52]
John Middlebrooks and David Green. 1991. Sound Localization by Human Listeners. Annual review of psychology 42 (02 1991), 135--59. https://doi.org/10.1146/annurev.ps.42.020191.001031
[53]
Allen William Mills. 1958. On the minimum audible angle. The Journal of the Acoustical Society of America 30, 4 (1958), 237--246.
[54]
Vimal Mollyn, Riku Arakawa, Mayank Goel, Chris Harrison, and Karan Ahuja. 2023. IMUPoser: Full-Body Pose Estimation Using IMUs in Phones, Watches, and Earbuds. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI'23). Association for Computing Machinery, New York, NY, USA, Article 529, 12 pages. https://doi.org/10.1145/3544548.3581392
[55]
Arinobu Niijima, Toki Takeda, Kentaro Tanaka, Ryosuke Aoki, and Yukio Koike. 2021. Reducing muscle activity when playing tremolo by using electrical muscle stimulation to learn efficient motor skills. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 3 (2021), 1--17.
[56]
Ryogo Niwa, Kazuya Izumi, Shieru Suzuki, and Yoichi Ochiai. 2022. EMS-Supported Throwing: Preliminary Investigation on EMS-Supported Training of Movement Form. In Universal Access in Human-Computer Interaction. Novel Design Approaches and Technologies: 16th International Conference, UAHCI 2022, Held as Part of the 24th HCI International Conference, HCII 2022, Virtual Event, June 26--July 1, 2022, Proceedings, Part I. Springer, 459--476.
[57]
Eva Ostertagova, Oskar Ostertag, and Jozef Kovác. 2014. Methodology and Application of the Kruskal-Wallis Test. Applied Mechanics and Materials 611 (08 2014), 115--120.
[58]
Max Pfeiffer, Tim Duente, and Michael Rohs. 2016. Let your body move: a prototyping toolkit for wearable force feedback with electrical muscle stimulation. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services. 418--427.
[59]
Kun Qian, Chenshu Wu, Zimu Zhou, Yue Zheng, Zheng Yang, and Yunhao Liu. 2017. Inferring motion direction using commodity wi-fi for interactive exergames. In Proceedings of the 2017 CHI conference on human factors in computing systems. 1961--1972.
[60]
Fazlay Rabbi, Taiwoo Park, Biyi Fang, Mi Zhang, and Youngki Lee. 2018. When virtual reality meets internet of things in the gym: Enabling immersive interactive machine exercises. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies 2, 2 (2018), 1--21.
[61]
T. Reilly. 2009. Ergonomics in Sport and Physical Activity. Human Kinetics. https://books.google.co.jp/books?id= rPJ6DwAAQBAJ
[62]
V.J.B. Rice. 1998. Ergonomics in Health Care and Rehabilitation. Elsevier Health Sciences. https://books.google.co.jp/ books?id=jDxtAAAAMAAJ
[63]
Brad J Schoenfeld and Jozo Grgic. 2020. Effects of range of motion on muscle development during resistance training interventions: A systematic review. SAGE open medicine 8 (2020), 2050312120901559.
[64]
Christian Schönauer, Kenichiro Fukushi, Alex Olwal, Hannes Kaufmann, and Ramesh Raskar. 2012. Multimodal motion guidance: techniques for adaptive and dynamic feedback. In Proceedings of the 14th ACM international conference on Multimodal interaction. ACM, New York, NY, USA, 133--140.
[65]
Konstantin Semionov and Iain McGregor. 2020. Effect of various spatial auditory cues on the perception of threat in a first-person shooter video game. In Proceedings of the 15th International Audio Mostly Conference. 22--29.
[66]
Seven. 2020. Seven. https://seven.app/.
[67]
Roland Sigrist, Georg Rauter, Robert Riener, and Peter Wolf. 2013. Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review. Psychonomic bulletin & review 20, 1 (2013), 21--53.
[68]
Alexa F Siu, Mike Sinclair, Robert Kovacs, Eyal Ofek, Christian Holz, and Edward Cutrell. 2020. Virtual reality without vision: A haptic and auditory white cane to navigate complex virtual worlds. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1--13.
[69]
Mel Slater, Bernhard Spanlang, Maria V Sanchez-Vives, and Olaf Blanke. 2010. First person experience of body transfer in virtual reality. PloS one 5, 5 (2010), e10564.
[70]
Rajinder Sodhi, Hrvoje Benko, and Andrew Wilson. 2012. LightGuide: projected visualizations for hand movement guidance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 179--188.
[71]
Maurício Sousa, João Vieira, Daniel Medeiros, Artur Arsenio, and Joaquim Jorge. 2016. SleeveAR: Augmented reality for rehabilitation using realtime feedback. In Proceedings of the 21st international conference on intelligent user interfaces. ACM, New York, NY, USA, 175--185.
[72]
Gabriele Spina, Guannan Huang, Anouk Vaes, Martijn Spruit, and Oliver Amft. 2013. COPDTrainer: a smartphone-based motion rehabilitation training system with real-time acoustic feedback. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing. ACM, New York, NY, USA, 597--606.
[73]
Thomas Stütz, Michael Domhardt, Gerlinde Emsenhuber, Daniela Huber, Martin Tiefengrabner, Nicholas Matis, and Simon Ginzinger. 2017. An interactive 3D health app with multimodal information representation for frozen shoulder. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, New York, NY, USA, 1--11.
[74]
Richard Tang, Xing-Dong Yang, Scott Bateman, Joaquim Jorge, and Anthony Tang. 2015. Physio@ Home: Exploring visual guidance and feedback techniques for physiotherapy exercises. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 4123--4132.
[75]
Yoshimasa Tokuyama, RPC Janaka Rajapakse, Sachiyo Yamabe, Kouichi Konno, and Yi-Ping Hung. 2019. A Kinect-Based Augmented Reality Game for Lower Limb Exercise. In 2019 International Conference on Cyberworlds (CW). IEEE, Long Beach, CA, USA, 399--402.
[76]
Milka Trajkova and Francesco Cafaro. 2018. Takes Tutu to ballet: designing visual and verbal feedback for augmented mirrors. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (2018), 1--30.
[77]
Hsin-Ruey Tsai, Yuan-Chia Chang, Tzu-Yun Wei, Chih-An Tsao, Xander Chin-yuan Koo, Hao-Chuan Wang, and Bing-Yu Chen. 2021. GuideBand: Intuitive 3D Multilevel Force Guidance on a Wristband in Virtual Reality. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1--13.
[78]
Eduardo Velloso, Andreas Bulling, and Hans Gellersen. 2013. Motionma: motion modelling and analysis by demonstration. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1309--1318.
[79]
H. Wallach. 1940. The Role of Head Movements and Vestibular and Visual Cues in Sound Localization. Journal of Experimental Psychology 27, 4 (1940), 339. https://doi.org/10.1037/h0054629
[80]
Zhu Wang, Anat Lubetzky, Marta Gospodarek, Makan TaghaviDilamani, and Ken Perlin. 2019. Virtual environments for rehabilitation of postural control dysfunction. arXiv preprint arXiv:1902.10223 (2019).
[81]
LaurenWard, Ben G Shirley, et al. 2019. Personalization in object-based audio for accessibility:Areviewof advancements for hearing impaired listeners. Journal of the Audio Engineering Society 67, 7/8 (2019), 584--597.
[82]
Robert Sessions Woodworth and Harold Schlosberg. 1954. Experimental psychology. Oxford and IBH Publishing.
[83]
Chengshuo Xia, Xinrui Fang, Riku Arakawa, and Yuta Sugiura. 2022. VoLearn: A Cross-Modal Operable Motion-Learning System Combined with Virtual Avatar and Auditory Feedback. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 2 (2022), 1--26.
[84]
Ungyeon Yang and Gerard Jounghyun Kim. 2002. Implementation and evaluation of ?just follow me?: An immersive, VR-based, motion-training system. Presence: Teleoperators & Virtual Environments 11, 3 (2002), 304--323.
[85]
Zhijian Yang, Yu-Lin Wei, Sheng Shen, and Romit Roy Choudhury. 2020. Ear-ar: indoor acoustic augmented reality on earphones. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking. 1--14.
[86]
Yuhang Zhao, Elizabeth Kupferstein, Hathaitorn Rojnirun, Leah Findlater, and Shiri Azenkot. 2020. The effectiveness of visual and audio wayfinding guidance on smartglasses for people with low vision. In Proceedings of the 2020 CHI conference on human factors in computing systems. ACM, New York, NY, USA, 1--14.
[87]
Huiyu Zhou and Huosheng Hu. 2008. Human motion tracking for rehabilitation?A survey. Biomedical signal processing and control 3, 1 (2008), 1--18.
[88]
Franz Zotter and Matthias Frank. 2019. Ambisonics: A practical 3D audio theory for recording, studio production, sound reinforcement, and virtual reality. Springer Nature.
Index Terms
- AudioMove: Applying the Spatial Audio to Multi-Directional Limb Exercise Guidance
Recommendations
Assisting the Multi-directional Limb Motion Exercise with Spatial Audio and Interactive Feedback
ISS Companion '23: Companion Proceedings of the 2023 Conference on Interactive Surfaces and SpacesGuiding users with limb motion exercises can assist them in their physical recovery and muscle training. However, traditional visual tracking methods often require multiple camera angles to help the user understand their movements and require the user ...
Sonification of a juggling performance using spatial audio
We developed a new musical instrument by equipping juggling balls with accelerometers, gyroscopes, and WiFi sensors. The system measures acceleration and rotation, allowing us to distinguish between ball movement and static states. We can determine the ...
Accelerometer & spatial audio technology: making touch-screen mobile devices accessible
ICCHP'10: Proceedings of the 12th international conference on Computers helping people with special needs: Part IAs mobile-phone design moves toward a touch-screen form factor, the visually disabled are faced with new accessibility challenges. The mainstream interaction model for touch-screen devices relies on the user having the ability to see spatially arranged ...
Comments
Information & Contributors
Information
Published In
September 2024
1136 pages
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 September 2024
Published in PACMHCI Volume 8, Issue MHCI
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Research Funds for the Central Universities Grant
- JST PRESTO
- KGRI Challenge Grant
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 237Total Downloads
- Downloads (Last 12 months)237
- Downloads (Last 6 weeks)58
Reflects downloads up to 25 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in