Abstract
Adding singing capabilities to a corpus-based concatenative text-to-speech (TTS) system can be addressed by explicitly collecting singing samples from the previously recorded speaker. However, this approach is only feasible if the considered speaker is also a singing talent. As an alternative, we consider appending a Harmonic plus Noise Model (HNM) speech-to-singing conversion module to a Unit Selection TTS (US-TTS) system. Two possible text-to-speech-to-singing synthesis approaches are studied: applying the speech-to-singing conversion to the US-TTS synthetic output, or implementing a hybrid US+HNM synthesis framework. The perceptual tests show that the speech-to-singing conversion yields similar singing resemblance than the natural version, but with lower naturalness. Moreover, no statistically significant differences are found between both strategies in terms of naturalness nor singing resemblance. Finally, the hybrid approach allows reducing more than twice the overall computational cost.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
The fstival speech synthesis system (2016). http://www.cstr.ed.ac.uk/projects/festival/
Babacan, O., Drugman, T., Raitio, T., Erro, D., Dutoit, T.: Parametric representation for singing voice synthesis: a comparative evaluation. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 2564–2568, May 2014
Bonada, J., Serra, X.: Synthesis of the singing voice by performance sampling and spectral models. IEEE Sig. Process. Mag. 24(2), 67–79 (2007)
Cen, L., Dong, M., Chan, P.: Template-based personalized singing voice synthesis. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 4509–4512 (2012)
Dong, M., Lee, S.W., Li, H., Chan, P., Peng, X., Ehnes, J.W., Huang, D.: I2R Speech2Singing perfects everyone’s singing. In: 15th Annual Conference of the International Speech Communication Association (INTERSPEECH), pp. 2148–2149 (2014)
Erro, D., Moreno, A., Bonafonte, A.: Flexible harmonic/stochastic speech synthesis. In: 6th ISCA Workshop on Speech Synthesis (SSW), Bonn, Germany, pp. 194–199, August 2007
Formiga, L., Trilla, A., Alías, F., Iriondo, I., Socoró, J.: Adaptation of the URL-TTS system to the 2010 Albayzin evaluation campaign. In: Proceedings of FALA 2010, Jornadas en Tecnología del Habla and Iberian SLTech Workshop, vol. 1, pp. 363–370, November 2020
Huber, S., Roebel, A.: On glottal source shape parameter transformation using a novel deterministic and stochastic speech analysis and synthesis system. In: 16th Annual Conference of the International Speech Communication Association (INTERSPEECH), pp. 289–293 (2015)
Kawahara, H., Masuda-Katsuse, I., De Cheveigné, A.: Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based F0 extraction: possible role of a repetitive structure in sounds. Speech Commun. 27(3), 187–207 (1999)
Kenmochi, H.: Singing synthesis as a new musical instrument. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 5385–5388 (2012)
Macon, M., Jensen-Link, L., George, E.: Concatenation-based MIDI-to-singing voice synthesis. In: 103rd Audio Engineering Society Convention, pp. 1–10 (1997)
Meron, Y.: High quality singing synthesis using the selection-based synthesis scheme. Ph.D. thesis, University of Tokyo (1999)
Nose, T., Kanemoto, M., Koriyama, T., Kobayashi, T.: HMM-based expressive singing voice synthesis with singing style control and robust pitch modeling. Comput. Speech Lang. 34(1), 308–322 (2015)
Oura, K., Mase, A.: Recent development of the HMM-based singing voice synthesis system-sinsy. In: 7th ISCA Workshop on Speech Synthesis (SSW), pp. 211–216 (2010)
Planet, S., Iriondo, I., Martínez, E., Montero, J.A.: TRUE: an online testing platform for multimedia evaluation. In: Workshop Corpora for Research on Emotion and Affect Marrakech, Morocco (2008)
Röbel, A., Fineberg, J.: Speech to chant transformation with the phase vocoder. In: 8th Annual Conference of the International Speech Communication Association (INTERSPEECH), pp. 2–3 (2007)
Roebel, A., Huber, S., Rodet, X., Degottex, G.: Analysis and modification of excitation source characteristics for singing voice synthesis. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 5381–5384, March 2012
Saitou, T., Goto, M., Unoki, M., Akagi, M.: Speech-to-singing synthesis: converting speaking voices to singing voices by controlling acoustic features unique to singing voices. In: IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), pp. 215–218 (2007)
Saitou, T., Goto, M., Unoki, M., Akagi, M.: Vocal conversion from speaking voice to singing voice using STRAIGHT. In: 8th Annual Conference of the International Speech Communication Association (INTERSPEECH), pp. 2–3 (2007)
Stylianou, Y.: Applying the harmonic plus noise model in concatenative speech synthesis. IEEE Trans. Speech Audio Process. 9(1), 21–29 (2001)
Sundberg, J.: The Science of the Singing Voice. Northern Illinois University Press, DeKalb (1987)
Sundberg, J.: The KTH synthesis of singing. Adv. Cogn. Psychol. 2(2), 131–143 (2006)
Acknowledgements
Marc Freixes thanks the support of the European Social Fund (ESF) and the Catalan Government (SUR/DEC) for the pre-doctoral FI grant No. 2016FI_B2 00094. This work has been partially funded by SUR/DEC (grant ref. 2014-SGR-0590). We also want to thank the people that took the perceptual test and Raúl Montaño for his help with the statistics.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Freixes, M., Socoró, J.C., Alías, F. (2016). Adding Singing Capabilities to Unit Selection TTS Through HNM-Based Conversion. In: Abad, A., et al. Advances in Speech and Language Technologies for Iberian Languages. IberSPEECH 2016. Lecture Notes in Computer Science(), vol 10077. Springer, Cham. https://doi.org/10.1007/978-3-319-49169-1_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-49169-1_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-49168-4
Online ISBN: 978-3-319-49169-1
eBook Packages: Computer ScienceComputer Science (R0)