Jochen Feitsch
ABSTRACT
This paper presents an interactive media installation that aims at providing users with the experience to sing like an opera singer from the 19th century. We designed a set of tangible and body-related interaction and feedback techniques and developed a singing voice synthesizer system that is controlled by the user's mouth shapes and gestures. This musical interface allows users to perform an aria without real singing. We adapted techniques from 3D body tracking, face recognition, singing voice synthesis, 3D rendering and tangible interaction to integrate them into an interactive musical interface.
- D. Butler, S. Izadi, O. Hilliges, D. Molyneaux, S. Hodges, and D. Kim. Shake'n'sense: Reducing interference for overlapping structured light depth cameras. In Proc. of the SIGCHI Conference on Human Factors in Computing Systems, CHI '12, pages 1933--1936, New York, NY, USA, 2012. ACM. Google Scholar
Digital Library
- P. Cano, A. Loscos, J. Bonada, M. D. Boer, and X. Serra. Voice morphing system for impersonating in karaoke applications. In In Proceedings of the ICMC, 2000.Google Scholar
- J. Cheng and P. Huang. Real-time mouth tracking and 3d reconstruction. In Image and Signal Processing (CISP), 2010 3rd International Congress on, volume 4, pages 1524--1528, 2010.Google ScholarCross Ref
- N. D'Alessandro, C. d'Alessandro, S. L. Beux, and B. Doval. Real-time calm synthesizer: New approaches in hands-controlled voice synthesis. In NIME, pages 266--271. IRCAM, 2006. Google ScholarDigital Library
- G. C. de Silva, T. Smyth, and M. J. Lyons. A novel face-tracking mouth controller and its application to interacting with bioacoustic models. In Y. Nagashima and M. J. Lyons, editors, NIME, pages 169--172. Shizuoka University of Art and Culture, 2004. Google ScholarDigital Library
- G. Fant. Acoustic Theory of Speech Production. Mouton De Gruyter, 1960.Google Scholar
- J. Feitsch, M. Strobel, and C. Geiger. Caruso - singen wie ein tenor. In Mensch & Computer Workshopband, pages 531--534, 2013.Google ScholarCross Ref
- J. Feitsch, M. Strobel, and C. Geiger. Singing like a tenor without a real voice. In Advances in Computer Entertainment, pages 258--269. Springer, 2013.Google ScholarDigital Library
- C. Geiger, H. Reckter, D. Paschke, F. Schulz, and C. Poepel. Towards Participatory Design and Evaluation of Theremin-based Musical Interfaces. In Proc. of the Int. Conference on New Interfaces for Musical Expression, pages 303--306, 2008.Google Scholar
- M. Goto, T. Nakano, S. Kajita, Y. Matsusaka, S. Nakaoka, and K. Yokoi. Voca-listener and voca-watcher: Imitating a human singer by using signal processing. In Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on, pages 5393--5396, 2012.Google ScholarCross Ref
- R. Linggard. Electronic Synthesis of Speech. Cambridge University Press, 1985. Google ScholarDigital Library
- M. J. Lyons, M. Haehnel, and N. Tetsutani. Designing, playing, and performing with a vision-based mouth interface. In Proceedings of the 2003 Conference on New Interfaces for Musical Expression, NIME '03, pages 116--121, Singapore, Singapore, 2003. National University of Singapore. Google ScholarDigital Library
- G. Odowichuk, S. Trail, P. Driessen, W. Nie, and W. Page. Sensor fusion: Towards a fully expressive 3d music control interface. In Communications, Computers and Signal Processing (PacRim), 2011 IEEE Pacific Rim Conference on, pages 836--841, 2011.Google ScholarCross Ref
- G. E. Peterson and H. L. Barney. Control methods used in a study of the vowels. The Journal of the Acoustical Society of America, 24(2):175--184, 1952.Google ScholarCross Ref
- X. Rodet. Synthesis and processing of the singing voice. In In Proc. 1st IEEE Benelux Workshop on Model based Processing and Coding of Audio (MPCA-2002, 2002.Google Scholar
- T. Yonezawa, N. Suzuki, K. Mase, and K. Kogure. Handysinger: Expressive singing voice morphing using personified hand-puppet interface. In Proc. of the Int. Conference on New Interfaces for Musical Expression, NIME '05, pages 121--126, Singapore, 2005. Google ScholarDigital Library
Index Terms
- Tangible and body-related interaction techniques for a singing voice synthesis installation
Recommendations
DeepSinger: Singing Voice Synthesis with Data Mined From the Web
KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data MiningIn this paper, we develop DeepSinger, a multi-lingual multi-singer singing voice synthesis (SVS) system, which is built from scratch using singing training data mined from music websites. The pipeline of DeepSinger consists of several steps, including ...
Singing Voice Database
Speech and ComputerAbstractThe first publicly available singing voice database, which was first released in 2012, is presented in this paper. This database contains recordings of professional singers including one Grammy Award winner. The database includes so-called plain ...
Rhythm Speech Lyrics Input for MIDI-Based Singing Voice Synthesis
PCM '09: Proceedings of the 10th Pacific Rim Conference on Multimedia: Advances in Multimedia Information ProcessingThis paper presents useful techniques and considerations in implementing underlying mandarin singing voice synthesis system using the RSLI unit. The system can receive the continuous speech of the lyrics of a song, and can synthesize the intended song ...
Comments