Abstract
Accurate statistical language models are needed, for example, for large vocabulary speech recognition. The construction of models that are computationally efficient and able to utilize long-term dependencies in the data is a challenging task. In this article we describe how a topical clustering obtained by ordered maps of document collections can be utilized for the construction of efficiently focusing statistical language models. Experiments on Finnish and English texts demonstrate that considerable improvements are obtained in perplexity compared to a general n-gram model and to manually classified topic categories. In the speech recognition task the recognition history and the current hypothesis can be utilized to focus the model towards the current discourse or topic, and then apply the focused model to re-rank the hypothesis.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
P. Clarkson and A. Robinson, “Language model adaptation using mixtures and an exponentially decaying cache,” In Proc. ICASSP, pp. 799–802, 1997.
R. Lau, R. Rosenfeld, and S. Roukos, “Trigger-based language models: A maximum entropy approach,” In Proc. ICASSP, pp. 45–48, 1993.
R.M. Iyer and M. Ostendorf, “Modelling long distance dependencies in language: Topic mixtures versus dynamic Cache model,” IEEE Trans. Speech and Audio Processing, 7, 1999.
D. Gildea and T. Hofmann, “Topic-based language modeling using EM,” In Proc. Eurospeech, pp. 2167–2170, 1999.
J. Bellegarda. “Exploiting latent semantic information in statistical language modeling,” Proc. IEEE, 88(8):1279–1296, 2000.
T. Kohonen, S. Kaski, K. Lagus, J. Salojärvi, V. Paatero, and A. Saarela. “Organization of a massive document collection,” IEEE Transactions on Neural Networks, 11(3):574–585, May 2000.
T. Kohonen. Self-Organizing Maps. Springer, Berlin, 2001. 3rd ed.
K. Lagus, “Text retrieval using self-organized document maps,” Neural Processing Letters, 2002. In press.
V. Siivola, M. Kurimo, and K. Lagus. “Large vocabulary statistical language modeling for continuous speech recognition,” In Proc. Eurospeech, 2001.
P. Clarkson and R. Rosenfeld, “Statistical language modeling using CMUCambridge toolkit,” in Proc. Eurospeech, pp. 2707–2710, 1997.
P. Clarkson and T. Robinson. “Improved language modelling through better language model evaluation measures,” Computer Speech and Language, 15(1):39–53, 2001.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kurimo, M., Lagus, K. (2002). An Efficiently Focusing Large Vocabulary Language Model. In: Dorronsoro, J.R. (eds) Artificial Neural Networks — ICANN 2002. ICANN 2002. Lecture Notes in Computer Science, vol 2415. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-46084-5_173
Download citation
DOI: https://doi.org/10.1007/3-540-46084-5_173
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44074-1
Online ISBN: 978-3-540-46084-8
eBook Packages: Springer Book Archive