Abstract
We have performed preliminary work on topological analysis of audio and text data for unsupervised speech processing. The work is based on the assumption that phoneme frequencies and contextual relationships are similar in the acoustic and text domains for the same language. Accordingly, this allowed the creation of a mapping between these spaces that takes into account their geometric structure. As a first step, generative methods based on variational autoencoders were chosen to map audio and text data into two latent vector spaces. In the next stage, persistent homology methods are used to analyze the topological structure of two spaces. Although the results obtained support the idea of the similarity of the two spaces, further research is needed to correctly map acoustic and text spaces, as well as to evaluate the real effect of including topological information in the autoencoder training process.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Peikert, R., Hauser, H., Carr, H., Fuchs, R. (eds.): Topological Methods in Data Analysis and Visualization II: Theory, Algorithms, and Applications. Springer-Verlag, p. 299 (2012). https://doi.org/10.1007/978-3-642-23175-9
Carlsson, G.: Topology and data. Bulletin (New Series) of the Amer. Math. Soc. 46, 255–308 (2009)
Zomorodian, A.J.: Topology for Computing. Cambridge University Press, p. 259 (2005)
Ghrist, R.: Barcodes: The Persistent Topology of Data. Bulletin (New Seires) of the Amer. Math. Soc. 45, 61–75 (2008)
Edelsbrunner, H., Harer, J.: Computational Topology, An Introduction. American Mathematical Society, p. 241 (2010)
Kaczynski, T., Mischaikow, K., Mrozek, M.: Computational Homology, p. 480. Springer (2004). https://doi.org/10.1007/b97315
Carlsson, E., Carlsson, G., de Silva, V.: An algebraic topological method for feature identification. Int. J. Comput. Geom. Appl. 16(4), 291–314 (2006)
Ferri, M., Frosini, P., Cerri, A., Di Fabio (eds.): Computational Topology in Image Context, p. 157. Springer (2012). https://doi.org/10.1007/978-3-642-30238-1
De Floriani, L., Spagnuolo, M.: Shape Analysis and Structuring, p. 296. Springer (2008). https://doi.org/10.1007/978-3-540-33265-7
Najman, L., Talbot, H.: Mathematical Morphology: From Theory to Applications, p. 503. John Wiley & Sons, Inc. (2010)
Edelsbrunner, H., Morozov, D.: Persistent homology: theory and practice. In: Proceedings of the European Congress of Mathematics, pp. 31–50 (2012)
Hsu, W., Zhang, Y., Glass, J.R.: Learning latent representations for speech generation and transformation (2017). arXiv:1704.04222, https://arxiv.org/abs/1704.04222
Kingma, D.P., Welling M.: Auto-encoding variational bayes (2013). arXiv:1312.6114, https://arxiv.org/abs/1312.6114
Garofolo, J.S., Lamel, L.F., Fisher, W.M., Fiscus, J.G., Pallett, D.S.: DARPA TIMIT acoustic-phonetic continous speech corpus. NIST speech disc 1–1.1. NASA STI/Recon technical report, 93, 27403 (1993)
McFee, B., Raffel, C., Liang, D., Ellis, D.P., McVicar, M., Battenberg, E., Nieto, O.: librosa: audio and music signal analysis in python. In: Proceedings of the 14th Python in Science Conference, vol. 8, pp. 18–25 (2015)
Srinivasa-Desikan, B.: Natural Language Processing and Computational Linguistics: A practical guide to text analysis with Python, Gensim, spaCy, and Keras. Packt Publishing Ltd (2018)
Bauer, U.: Ripser: efficient computation of Vietoris-Rips persistence barcodes. J. Appl. Comput. Topology 5, 391–423 (2021)
Acknowledgements
The work is supported by the Ministry of Education and Science of the Republic of Kazakhstan under the grants No. AP13068635 and No. AP08053085.
We also thank Dr. Nikolay Makarenko from The Central Astronomical Observatory of the Russian Academy of Sciences at Pulkovo for his invaluable comments and lecture notes on the topic of topological data analysis.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Yessenbayev, Z., Kozhirbayev, Z. (2022). Comparison of Word Embeddings of Unaligned Audio and Text Data Using Persistent Homology. In: Prasanna, S.R.M., Karpov, A., Samudravijaya, K., Agrawal, S.S. (eds) Speech and Computer. SPECOM 2022. Lecture Notes in Computer Science(), vol 13721. Springer, Cham. https://doi.org/10.1007/978-3-031-20980-2_59
Download citation
DOI: https://doi.org/10.1007/978-3-031-20980-2_59
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20979-6
Online ISBN: 978-3-031-20980-2
eBook Packages: Computer ScienceComputer Science (R0)