Yudai Tanaka
Jun Nishida
Pedro Lopes
ABSTRACT
We propose a novel interface concept in which interactive systems directly manipulate the user's head orientation. We implement this using electrical-muscle-stimulation (EMS) of the neck muscles, which turns the head around its yaw (left/right) and pitch (up/down) axis. As the first exploration of EMS for head actuation, we characterized which muscles can be robustly actuated. Second, we evaluated the accuracy of our system for actuating participants' head orientation towards static targets and trajectories. Third, we demonstrated how it enables interactions not possible before by building a range of applications, such as (1) synchronizing head orientations of two users, which enables a user to communicate head nods to another user while listening to music, and (2) directly changing the user's head orientation to locate objects in AR. Finally, in our second study, participants felt that our head actuation contributed positively to their experience in four distinct applications.
Supplemental Material
- Ardouin, J., Lécuyer, A., Marchal, M., Riant, C. and Marchand, E. 2012. FlyVIZ: a novel display device to provide humans with 360 vision by coupling catadioptric camera with hmd. Proceedings of the 18th ACM symposium on Virtual reality software and technology (New York, NY, USA, Dec. 2012), 41–44.Google Scholar
Digital Library
- Ben Abdallah, I., Bouteraa, Y. and Rekik, C. 2017. Design and development of 3d printed myoelectric robotic exoskeleton for hand rehabilitation. International Journal on Smart Sensing and Intelligent Systems. 10, (Jun. 2017), 341–366. DOI:https://doi.org/10.21307/ijssis-2017-215.Google ScholarCross Ref
- Bézier Path Creator | Utilities Tools | Unity Asset Store: https://assetstore.unity.com/packages/tools/utilities/b-zier-path-creator-136082. Accessed: 2021-04-07.Google Scholar
- Bhatt, A.D., Goodwin, N., Cash, E., Bhatt, G., Silverman, C.L., Spanos, W.J., Bumpous, J.M., Potts, K., Redman, R., Allison, W.A. and Dunlap, N.E. 2015. Impact of transcutaneous neuromuscular electrical stimulation on dysphagia in patients with head and neck cancer treated with definitive chemoradiation. Head & Neck. 37, 7 (2015), 1051–1056. DOI:https://doi.org/10.1002/hed.23708.Google ScholarCross Ref
- Core Motion | Apple Developer Documentation: https://developer.apple.com/documentation/coremotion. Accessed: 2021-04-07.Google Scholar
- CyberGrasp: http://www.cyberglovesystems.com/cybergrasp. Accessed: 2021-03-18.Google Scholar
- Fan, K., Huber, J., Nanayakkara, S. and Inami, M. 2014. SpiderVision: extending the human field of view for augmented awareness. Proceedings of the 5th Augmented Human International Conference (New York, NY, USA, Mar. 2014), 1–8.Google ScholarDigital Library
- Fight or Flight | Fire Prevention Services | The University of Texas at Austin: https://fireprevention.utexas.edu/firesafety/fight-or-flight. Accessed: 2021-09-05.Google Scholar
- Frangos, E. and Komisaruk, B.R. 2017. Access to Vagal Projections via Cutaneous Electrical Stimulation of the Neck: fMRI Evidence in Healthy Humans. Brain Stimulation. 10, 1 (Jan. 2017), 19–27. DOI:https://doi.org/10.1016/j.brs.2016.10.008.Google ScholarCross Ref
- Freed, M., Freed, L., Chatburn, R. and Christian, M. 2001. Electrical Stimulation for swallowing disorders caused by stroke. Respiratory care. 46, (May 2001), 466–74.Google Scholar
- Frisoli, A., Montagner, A., Borelli, L., Salsedo, F. and Bergamasco, M. 2009. A Force-Feedback Exoskeleton for Upper-Limb Rehabilitation in Virtual Reality. Applied Bionics and Biomechanics. 6, (Jul. 2009), 115–126. DOI:https://doi.org/10.1080/11762320902959250.Google ScholarCross Ref
- Gruenefeld, U., Stratmann, T.C., Ali, A.E., Boll, S. and Heuten, W. 2018. RadialLight: exploring radial peripheral LEDs for directional cues in head-mounted displays. Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services (New York, NY, USA, Sep. 2018), 1–6.Google ScholarDigital Library
- Gruenefeld, U., Stratmann, T.C., Prädel, L. and Heuten, W. 2018. MonoculAR: a radial light display to point towards out-of-view objects on augmented reality devices. Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (New York, NY, USA, Sep. 2018), 16–22.Google ScholarDigital Library
- Gugenheimer, J., Wolf, D., Eiriksson, E.R., Maes, P. and Rukzio, E. 2016. GyroVR: Simulating Inertia in Virtual Reality using Head Worn Flywheels. Proceedings of the 29th Annual Symposium on User Interface Software and Technology (New York, NY, USA, Oct. 2016), 227–232.Google ScholarDigital Library
- Gugenheimer, J., Wolf, D., Haas, G., Krebs, S. and Rukzio, E. 2016. SwiVRChair: A Motorized Swivel Chair to Nudge Users’ Orientation for 360 Degree Storytelling in Virtual Reality. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, May 2016), 1996–2000.Google ScholarDigital Library
- Guinet, A.L., Bouyer, G., Otmane, S. and Desailly, E. 2019. Reliability of the head tracking measured by Microsoft Hololens during different walking conditions. Computer Methods in Biomechanics and Biomedical Engineering. 22, sup1 (Oct. 2019), S169–S171. DOI:https://doi.org/10.1080/10255842.2020.1714228.Google ScholarCross Ref
- Hassan, M., Daiber, F., Wiehr, F., Kosmalla, F. and Krüger, A. 2017. FootStriker: An EMS-based Foot Strike Assistant for Running. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 1, 1 (Mar. 2017), 2:1-2:18. DOI:https://doi.org/10.1145/3053332.Google ScholarDigital Library
- Higuchi, K. and Rekimoto, J. 2013. Flying head: a head motion synchronization mechanism for unmanned aerial vehicle control. CHI ’13 Extended Abstracts on Human Factors in Computing Systems (New York, NY, USA, Apr. 2013), 2029–2038.Google Scholar
- Holographic Remoting Player - Mixed Reality: https://docs.microsoft.com/en-us/windows/mixed-reality/develop/platform-capabilities-and-apis/holographic-remoting-player. Accessed: 2021-04-07.Google Scholar
- HoloLens 2—Overview, Features, and Specs | Microsoft HoloLens: https://www.microsoft.com/en-us/hololens/hardware. Accessed: 2021-04-07.Google Scholar
- Jordan, J., Heusser, K., Brinkmann, J. and Tank, J. 2012. Electrical carotid sinus stimulation in treatment resistant arterial hypertension. Autonomic Neuroscience. 172, 1 (Dec. 2012), 31–36. DOI:https://doi.org/10.1016/j.autneu.2012.10.009.Google ScholarCross Ref
- Kamibayashi, L.K. and Richmond, F.J. 1998. Morphometry of human neck muscles. Spine. 23, 12 (Jun. 1998), 1314–1323. DOI:https://doi.org/10.1097/00007632-199806150-00005.Google ScholarCross Ref
- Karnath, H.-O. 1995. Transcutaneous electrical stimulation and vibration of neck muscles in neglect. Experimental Brain Research. 105, 2 (Aug. 1995), 321–324. DOI:https://doi.org/10.1007/BF00240969.Google ScholarCross Ref
- Kasahara, S., Nishida, J. and Lopes, P. 2019. Preemptive Action: Accelerating Human Reaction using Electrical Muscle Stimulation Without Compromising Agency. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, May 2019), 1–15.Google ScholarDigital Library
- Kasahara, S. and Rekimoto, J. 2014. JackIn: integrating first-person view with out-of-body vision generation for human-human augmentation. Proceedings of the 5th Augmented Human International Conference (New York, NY, USA, Mar. 2014), 1–8.Google ScholarDigital Library
- Kaul, O.B., Pfeiffer, M. and Rohs, M. 2016. Follow the Force: Steering the Index Finger towards Targets using EMS. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (New York, NY, USA, May 2016), 2526–2532.Google ScholarDigital Library
- Kon, Y., Nakamura, T. and Kajimoto, H. 2017. HangerOVER: HMD-embedded haptics display with hanger reflex. ACM SIGGRAPH 2017 Emerging Technologies (New York, NY, USA, Jul. 2017), 1–2.Google ScholarDigital Library
- Kono, M., Takahashi, T., Nakamura, H., Miyaki, T. and Rekimoto, J. 2018. Design Guideline for Developing Safe Systems that Apply Electricity to the Human Body. ACM Transactions on Computer-Human Interaction. 25, 3 (Jun. 2018), 1–36. DOI:https://doi.org/10.1145/3184743.Google ScholarDigital Library
- Langmore, S.E., McCulloch, T.M., Krisciunas, G.P., Lazarus, C.L., Daele, D.J.V., Pauloski, B.R., Rybin, D. and Doros, G. 2016. Efficacy of electrical stimulation and exercise for dysphagia in patients with head and neck cancer: A randomized clinical trial. Head & Neck. 38, S1 (2016), E1221–E1231. DOI:https://doi.org/10.1002/hed.24197.Google ScholarCross Ref
- Lin, Y.-C., Chang, Y.-J., Hu, H.-N., Cheng, H.-T., Huang, C.-W. and Sun, M. 2017. Tell Me Where to Look: Investigating Ways for Assisting Focus in 360 Video. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, May 2017), 2535–2545.Google ScholarDigital Library
- Lin, Y.-T., Liao, Y.-C., Teng, S.-Y., Chung, Y.-J., Chan, L. and Chen, B.-Y. 2017. Outside-In: Visualizing Out-of-Sight Regions-of-Interest in a 360 Video Using Spatial Picture-in-Picture Previews. Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (New York, NY, USA, Oct. 2017), 255–265.Google ScholarDigital Library
- Liu, S.-H., Yen, P.-C., Mao, Y.-H., Lin, Y.-H., Chandra, E. and Chen, M.Y. 2020. HeadBlaster: a wearable approach to simulating motion perception using head-mounted air propulsion jets. ACM Transactions on Graphics. 39, 4 (Jul. 2020), 84:84:1-84:84:12. DOI:https://doi.org/10.1145/3386569.3392482.Google ScholarDigital Library
- Lopes, P., Ion, A. and Baudisch, P. 2015. Impacto: Simulating Physical Impact by Combining Tactile Stimulation with Electrical Muscle Stimulation. Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (New York, NY, USA, Nov. 2015), 11–19.Google ScholarDigital Library
- Lopes, P., Ion, A., Mueller, W., Hoffmann, D., Jonell, P. and Baudisch, P. 2015. Proprioceptive Interaction. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (New York, NY, USA, Apr. 2015), 939–948.Google Scholar
- Lopes, P., Jonell, P. and Baudisch, P. 2015. Affordance++: Allowing Objects to Communicate Dynamic Use. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (New York, NY, USA, Apr. 2015), 2515–2524.Google ScholarDigital Library
- Lopes, P., You, S., Cheng, L.-P., Marwecki, S. and Baudisch, P. 2017. Providing Haptics to Walls & Heavy Objects in Virtual Reality by Means of Electrical Muscle Stimulation. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery. 1471–1482.Google Scholar
- Lopes, P., You, S., Ion, A. and Baudisch, P. 2018. Adding Force Feedback to Mixed Reality Experiences and Games using Electrical Muscle Stimulation. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, Apr. 2018), 1–13.Google ScholarDigital Library
- Lopes, P., Yüksel, D., Guimbretière, F. and Baudisch, P. 2016. Muscle-plotter: An Interactive System based on Electrical Muscle Stimulation that Produces Spatial Output. Proceedings of the 29th Annual Symposium on User Interface Software and Technology (New York, NY, USA, Oct. 2016), 207–217.Google ScholarDigital Library
- Maayah, M. and Al-Jarrah, M. 2010. Evaluation of Transcutaneous Electrical Nerve Stimulation as a Treatment of Neck Pain due to Musculoskeletal Disorders. Journal of Clinical Medicine Research. 2, 3 (Jun. 2010), 127–136. DOI:https://doi.org/10.4021//jocmr.v2i3.406.Google Scholar
- Maeda, T., Ando, H., Amemiya, T., Nagaya, N., Sugimoto, M. and Inami, M. 2005. Shaking the world: galvanic vestibular stimulation as a novel sensation interface. ACM SIGGRAPH 2005 Emerging technologies (New York, NY, USA, Jul. 2005), 17-es.Google Scholar
- Maekawa, A., Matsubara, S., Wakisaka, S., Uriu, D., Hiyama, A. and Inami, M. 2020. Dynamic Motor Skill Synthesis with Human-Machine Mutual Actuation. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, Apr. 2020), 1–12.Google ScholarDigital Library
- Marieb, E.N. and Hoehn, K. 2007. Human anatomy & physiology. Pearson Benjamin Cummings.Google Scholar
- Matsuda, A., Nozawa, K., Takata, K., Izumihara, A. and Rekimoto, J. 2020. HapticPointer: A Neck-worn Device that Presents Direction by Vibrotactile Feedback for Remote Collaboration Tasks. Proceedings of the Augmented Humans International Conference (New York, NY, USA, Mar. 2020), 1–10.Google ScholarDigital Library
- Medrano, S.N., Pfeiffer, M. and Kray, C. 2020. Remote Deictic Communication: Simulating Deictic Pointing Gestures across Distances Using Electro Muscle Stimulation. International Journal of Human–Computer Interaction. 36, 19 (Nov. 2020), 1867–1882. DOI:https://doi.org/10.1080/10447318.2020.1801171.Google ScholarCross Ref
- Nagai, K., Tanoue, S., Akahane, K. and Sato, M. 2015. Wearable 6-DoF wrist haptic device “SPIDAR-W.” SIGGRAPH Asia 2015 Haptic Media And Contents Design (New York, NY, USA, Nov. 2015), 1–2.Google Scholar
- Nishida, J. and Suzuki, K. 2017. bioSync: A Paired Wearable Device for Blending Kinesthetic Experience. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, May 2017), 3316–3327.Google ScholarDigital Library
- Nith, R., Teng, S.-Y., Li, P., Tao, Y. and Lopes, P. 2021. DextrEMS: Achieving Dexterity in Electrical Muscle Stimulation by Combining it with Brakes. Proceedings of the 34th Annual Symposium on User Interface Software and Technology (Virtual Event USA, Oct. 2021).Google Scholar
- Pfeiffer, M., Dünte, T., Schneegass, S., Alt, F. and Rohs, M. 2015. Cruise Control for Pedestrians: Controlling Walking Direction using Electrical Muscle Stimulation. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (New York, NY, USA, Apr. 2015), 2505–2514.Google ScholarDigital Library
- Pizzamiglio, L., Vallar, G. and Magnotti, L. 1996. Transcutaneous electrical stimulation of the neck muscles and hemineglect rehabilitation. Restorative Neurology and Neuroscience. 10, 4 (Jan. 1996), 197–203. DOI:https://doi.org/10.3233/RNN-1996-10402.Google ScholarCross Ref
- RehaMove_Katalog_englisch_2017-02_Web: https://hasomed.de/wp-content/uploads/hasomed-fileadmin/RehaMove/Mediathek/Broschueren_Flyer/RehaMove_Katalog_englisch_2017-02_Web.pdf. Accessed: 2021-12-23.Google Scholar
- Ryu, J.S., Kang, J.Y., Park, J.Y., Nam, S.Y., Choi, S.H., Roh, J.L., Kim, S.Y. and Choi, K.H. 2009. The effect of electrical stimulation therapy on dysphagia following treatment for head and neck cancer. Oral Oncology. 45, 8 (Aug. 2009), 665–668. DOI:https://doi.org/10.1016/j.oraloncology.2008.10.005.Google ScholarCross Ref
- Schaack, S., Chernyshov, G., Ragozin, K., Tag, B., Peiris, R. and Kunze, K. 2019. Haptic Collar: Vibrotactile Feedback around the Neck for Guidance Applications. Proceedings of the 10th Augmented Human International Conference 2019 (New York, NY, USA, Mar. 2019), 1–4.Google ScholarDigital Library
- Schafer, R.C. 1987. Clinical Biomechanics: Musculoskeletal Actions and Reactions. Williams & Wilkins.Google Scholar
- Schoop, E., Smith, J. and Hartmann, B. 2018. HindSight: Enhancing Spatial Awareness by Sonifying Detected Objects in Real-Time 360-Degree Video. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, Apr. 2018), 1–12.Google ScholarDigital Library
- Shin, J.-G., Onchi, E., Reyes, M.J., Song, J., Lee, U., Lee, S.-H. and Saakes, D. 2019. Slow Robots for Unobtrusive Posture Correction. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, May 2019), 1–10.Google ScholarDigital Library
- Sra, M., Jain, A. and Maes, P. 2019. Adding Proprioceptive Feedback to Virtual Reality Experiences Using Galvanic Vestibular Stimulation. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery. 1–14.Google Scholar
- Takahashi, A., Brooks, J., Kajimoto, H. and Lopes, P. 2021. Increasing Electrical Muscle Stimulation's Dexterity by means of Back of the Hand Actuation. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, May 2021), 1–12.Google ScholarDigital Library
- Tamaki, E., Miyaki, T. and Rekimoto, J. 2010. PossessedHand: a hand gesture manipulation system using electrical stimuli. Proceedings of the 1st Augmented Human International Conference (New York, NY, USA, Apr. 2010), 1–5.Google ScholarDigital Library
- Tamaki, E., Miyaki, T. and Rekimoto, J. 2011. PossessedHand: techniques for controlling human hands using electrical muscles stimuli. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (New York, NY, USA, May 2011), 543–552.Google ScholarDigital Library
- Tsetserukou, D., Sato, K. and Tachi, S. 2010. ExoInterfaces: novel exosceleton haptic interfaces for virtual reality, augmented sport and rehabilitation. Proceedings of the 1st Augmented Human International Conference (New York, NY, USA, Apr. 2010), 1–6.Google ScholarDigital Library
- Vallar, G., Rusconi, M.L., Barozzi, S., Bernardini, B., Ovadia, D., Papagno, C. and Cesarani, A. 1995. Improvement of left visuo-spatial hemineglect by left-sided transcutaneous electrical stimulation. Neuropsychologia. 33, 1 (Jan. 1995), 73–82. DOI:https://doi.org/10.1016/0028-3932(94)00088-7.Google ScholarCross Ref
- Wu, D., Wang, L. and Li, P. 2016. A 6-DOF exoskeleton for head and neck motion assist with parallel manipulator and sEMG based control. 2016 International Conference on Control, Decision and Information Technologies (CoDIT) (Apr. 2016), 341–344.Google ScholarCross Ref
- Xiao, R. and Benko, H. 2016. Augmenting the Field-of-View of Head-Mounted Displays with Sparse Peripheral Displays. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, May 2016), 1221–1232.Google ScholarDigital Library
- Yamazaki, Y., Hasegawa, S., Mitake, H. and Shirai, A. 2019. Neck strap haptics: an algorithm for non-visible VR information using haptic perception on the neck. ACM SIGGRAPH 2019 Posters (New York, NY, USA, Jul. 2019), 1–2.Google ScholarDigital Library
- Yang, J., Sasikumar, P., Bai, H., Barde, A., Sörös, G. and Billinghurst, M. 2020. The effects of spatial auditory and visual cues on mixed reality remote collaboration. Journal on Multimodal User Interfaces. 14, 4 (Dec. 2020), 337–352. DOI:https://doi.org/10.1007/s12193-020-00331-1.Google ScholarCross Ref
- Yem, V., Vu, K., Kon, Y. and Kajimoto, H. 2018. Effect of Electrical Stimulation Haptic Feedback on Perceptions of Softness-Hardness and Stickiness While Touching a Virtual Object. 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (Mar. 2018), 89–96.Google ScholarCross Ref
- Zhang, H. and Agrawal, S. 2017. An Active Neck Brace Controlled by a Joystick to Assist Head Motion. IEEE Robotics and Automation Letters. PP, (Jul. 2017), 1–1. DOI:https://doi.org/10.1109/LRA.2017.2728858.Google ScholarCross Ref
- Zhang, H., Albee, K. and Agrawal, S.K. 2018. A spring-loaded compliant neck brace with adjustable supports. Mechanism and Machine Theory. 125, (Jul. 2018), 34–44. DOI:https://doi.org/10.1016/j.mechmachtheory.2017.12.025.Google ScholarCross Ref
- Zhang, H., Chang, B.-C., Andrews, J., Mitsumoto, H. and Agrawal, S. 2019. A robotic neck brace to characterize head-neck motion and muscle electromyography in subjects with amyotrophic lateral sclerosis. Annals of Clinical and Translational Neurology. 6, 9 (2019), 1671–1680. DOI:https://doi.org/10.1002/acn3.50864.Google ScholarCross Ref
- Zhang, J., Fiers, P., Witte, K.A., Jackson, R.W., Poggensee, K.L., Atkeson, C.G. and Collins, S.H. 2017. Human-in-the-loop optimization of exoskeleton assistance during walking. Science. 356, 6344 (Jun. 2017), 1280–1284. DOI:https://doi.org/10.1126/science.aal5054.Google ScholarCross Ref
Index Terms
- Electrical Head Actuation: Enabling Interactive Systems to Directly Manipulate Head Orientation
Index terms have been assigned to the content through auto-classification.
Recommendations
DextrEMS: Increasing Dexterity in Electrical Muscle Stimulation by Combining it with Brakes
UIST '21: The 34th Annual ACM Symposium on User Interface Software and TechnologyElectrical muscle stimulation (EMS) is an emergent technique that miniaturizes force feedback, especially popular for untethered haptic devices, such as mobile gaming, VR, or AR. However, the actuation displayed by interactive systems based on EMS is ...
Demonstrating Electrical Head Actuation: Enabling Interactive Systems to Directly Manipulate Head Orientation
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing SystemsWe demonstrate a novel interface concept in which interactive systems directly manipulate the user's head orientation. We implement this using electrical-muscle-stimulation (EMS) of the neck muscles, which turns the head around its yaw (left/right) and ...
Eye&Head: Synergetic Eye and Head Movement for Gaze Pointing and Selection
UIST '19: Proceedings of the 32nd Annual ACM Symposium on User Interface Software and TechnologyEye gaze involves the coordination of eye and head movement to acquire gaze targets, but existing approaches to gaze pointing are based on eye-tracking in abstraction from head motion. We propose to leverage the synergetic movement of eye and head, and ...
Comments