Hiroya Miura
ABSTRACT
The fusion of music-information processing and human physical functions will enable new musical experiences to be created. We developed an interactive music system, which provides users with the experience of conducting a musical performance. This system converts music arranged using melody morphing method based on the general theory of tonal music into electrical muscle stimulation to control the body movements of multiple performers (e.g., musicians and dancers) via devices attached to the performers’ hands and feet. The melodies used in the system are divided into segments, and each segment has multiple variations of melodies. The user can interactively control the performers, thus the actual performance.
Supplemental Material
- Ayaka Ebisu, Satoshi Hashizume, and Yoichi Ochiai. 2018. Building a feedback loop between electrical stimulation and percussion learning. In ACM SIGGRAPH 2018 Studio(SIGGRAPH ’18). ACM, 1–2.Google Scholar
Digital Library
- Masatoshi Hamanaka, Keiji Hirata, and Satoshi Tojo. 2007. Implementing ‘A Generating Theory of Tonal Music’. Journal of New Music Research 35, 4 (2007), 249–277.Google ScholarCross Ref
- Masatoshi Hamanaka, Keiji Hirata, and Satoshi Tojo. 2008. Melody Morphing Method Based on GTTM. In Proceedings of the 2008 International Computer Music Conference(ICMC ’08). 155–158.Google Scholar
- Masatoshi Hamanaka, Keiji Hirata, and Satoshi Tojo. 2017. deepGTTM-III: Multi-task Learning with Grouping and Metrical Structures. In Proceedings of the 2017 International Symposium on Computer Music Multidisciplinary Research(CMMR ’17). 238–251.Google Scholar
- Yoichiro Kawaguchi. 2013. Squarepusher × Z-MACHINES. Retrieved February 24, 2022 from https://youtu.be/VkUq4sO4LQMGoogle Scholar
- Fred Lerdahl and linguist Ray Jackendoff. 1983. A Generative Theory of Tonal Music.Cambridge, Massachusetts: MIT Press.Google Scholar
- Pedro Lopes, Max Pfeiffer, Michael Rohs, and Patrick Baudisch. 2016. Hands-on Introduction to Interactive Electrical Muscle Stimulation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’16). ACM, 944–947.Google ScholarDigital Library
- Pat Metheny. 2013. Orchestrion - An Excerpt from The Orchestrion Project. Retrieved February 24, 2022 from https://youtu.be/evHVh4bqaOQGoogle Scholar
- Jun Nishida and Kenji Suzuki. 2017. bioSync: A Paired Wearable Device for Blending Kinesthetic Experience. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’17). ACM, 3316–3327.Google ScholarDigital Library
- Nobuhiro Takahashi, Hayato Takahashi, and Hideki Koike. 2019. Soft Exoskeleton Glove Enabling Force Feedback for Human-Like Finger Posture Control with 20 Degrees of Freedom. In Proceedings of IEEE World Haptics Conference(WHC ’19). IEEE, 217–222.Google ScholarCross Ref
- Emi Tamaki, Miyaki Takashi, and Jun Rekimoto. 2011. Possessedhand: Techniques for controlling human hands using electrical muscles stimuli. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’11). ACM, 543–552.Google ScholarDigital Library
Recommendations
Stimulated percussions: method to control human for learning music by using electrical muscle stimulation
AH '17: Proceedings of the 8th Augmented Human International ConferenceIn musical performances, it is important to produce rhythms correctly. However, when beginners play musical instruments, it can be difficult for them to understand rhythms using only visual and auditory rhythm information. To solve this problem, we ...
Stimulated percussions: techniques for controlling human as percussive musical instrument by using electrical muscle stimulation
SA '16: SIGGRAPH ASIA 2016 PostersWe propose Stimulated Percussions, a new method designed to beat out rhythm by controlling human bodies with percussions using electrical muscle stimulation (EMS). Numerous studies in the fields of both science and art have been carried out to expand ...
Augmented Percussionists: A New Musical Experience by Fusion of Melody Morphing and Electrical Muscle Stimulation
AH2021: 12th Augmented Human International ConferenceThe fusion of music information processing and human physical functions will allow new musical experiences to be created. We developed an interactive music system called “Augmented Percussionists,” which provides users with the experience of playing an ...
Comments