Skip to main content

WaVAEtable Synthesis

  • Conference paper
  • First Online:
Music in the AI Era (CMMR 2021)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13770 ))

Included in the following conference series:

  • 633 Accesses

Abstract

Timbral autoencoders, a class of generative model that learn the timbre distribution of audio data, are a current research focus in music technology; however, despite recent improvements, they have rarely been used in music composition or musical systems due to issues of static musical output, general lack of real-time synthesis and the unwieldiness of synthesis parameters. This project proposes a solution to these issues by combining timbral autoencoder models with a classic computer music synthesis technique in wavetable synthesis. A proof-of-concept implementation in Python, with controllers in Max and SuperCollider, demonstrates the timbral autoencoder’s capability as a wavetable generator. This concept is generally architecture agnostic, showing that most existing timbral autoencoders could be adapted for use in real-time music creation today, regardless of their capabilities for real-time synthesis and time-varying timbre.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bristow-Johnson, R.: Wavetable synthesis 101, a fundamental perspective. In: Proceedings 101st Convention of the Audio Engineering Society (1996)

    Google Scholar 

  2. Colonel, J., Curro, C., Keene, S.: Autoencoding neural networks as musical audio synthesizers. In: Proceedings of the 21st International Conference on Digital Audio Effects (2018)

    Google Scholar 

  3. Engel, J., et al.: Neural audio synthesis of musical notes with WaveNet autoencoders. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70 (2017)

    Google Scholar 

  4. Esling, P., Chemla-Romeu-Santos, A., Bitton, A.: Generative timbre spaces: regularizing variational auto-encoders with perceptual metrics. In: Proceedings of the 21st International Conference on Digital Audio Effects (2018)

    Google Scholar 

  5. Griffin, D., Lim, J.: Signal estimation from modified short-time Fourier transform. IEEE Trans. Acoust. Speech Signal Process. 32(2), 236–243 (1984)

    Article  Google Scholar 

  6. Hantrakul, L., Yang, L.C.: Neural wavetable: a playable wavetable synthesizer using neural networks. In: Workshop on Machine Learning for Creativity and Design (2018)

    Google Scholar 

  7. Huzaifah, M., Wyse, L.: Deep generative models for musical audio synthesis. In: Miranda, E.R. (ed.) Handbook of Artificial Intelligence for Music, pp. 639–678. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72116-9_22

    Chapter  Google Scholar 

  8. Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint, arXiv:1312.6114 (2013)

  9. Luo, Y.J., Cheuk, K.W., Nakano, T., Goto, M., Herremans, D.: Unsupervised disentanglement of pitch and timbre for isolated musical instrument sounds. In: Proceedings of the 2020 International Society of Music Information Retrieval Conference (2020)

    Google Scholar 

  10. Mauch, M., Dixon, S.: pYIN: a fundamental frequency estimator using probabilistic threshold distributions. In: Proceedings of the 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (2014)

    Google Scholar 

  11. Paszke, A., et al.: Automatic differentiation in PyTorch. In: Proceedings of the 31st Conference on Neural Information Processing Systems (2017)

    Google Scholar 

Download references

Acknowledgments

Special thanks to Karl Yerkes (MAT, University of California Santa Barbara) for his great help with SuperCollider and OSC implementations.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jeremy Hyrkas .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hyrkas, J. (2023). WaVAEtable Synthesis. In: Aramaki, M., Hirata, K., Kitahara, T., Kronland-Martinet, R., Ystad, S. (eds) Music in the AI Era. CMMR 2021. Lecture Notes in Computer Science, vol 13770 . Springer, Cham. https://doi.org/10.1007/978-3-031-35382-6_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35382-6_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35381-9

  • Online ISBN: 978-3-031-35382-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics