Abstract
In Indian Languages, root words will be either combined or modified to match the context with reference to tense, number and/or gender. So the number of unique words will increase when compared to many European languages. Whatever be the size of the text corpus used for language modeling cannot contain all the possible inflected words. A word which occurred during testing but not in training data is called Out of Vocabulary (OOV) word. Similarly, the text corpus cannot have all possible sequence of words. So Due to this data sparsity, Automatic Speech Recognition system (ASR) may not accommodate all the words in the language model/irrespective of the size of the text corpus. It also becomes computationally challenging if the volume of the data increases exponentially due to morphological changes to the root word. To reduce the OOVs in the language model, a new unsupervised stemming method is proposed in this paper for one Indian language, Telugu, based on the method proposed for Hindi. Other issues in the language modeling for Telugu using techniques like smoothing and interpolation, with supervised and unsupervised stemming data is also analyzed. It is observed that the smoothing techniques Witten–Bell and Kneser–Ney performing well when compared to other techniques, on pre-processed data with supervised learning. The ASRs accuracy is improved by 0.76% and 0.94% with supervised and unsupervised stemming respectively.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anumanchipalli, G., et al. (2005). Development of Indian Language Speech Databases for Large Vocabulary Speech Recognition Systems. In: Proceedings on SPECOM, 2005.
Basha Shaik, M. A., Rybach, D., Hahn, S., Schlüter, R., & Ney, H. (2012). Hierarchical Hybrid Language models for Open Vocabulary Continuous Speech Recognition using WFST. In: Proceedings of Working Statuary Percept Audit (SAPA – SCALE) (pp. 46–51).
Brown, P. F., Della Pietra, V. J., Mercer, R. L., Della Pietra, S. A., & Lai, J. C. (1992). An estimate of an upper bound for the entropy of English. Computational Linguistics, 18(1), 31–40.
Chen, S., Beeferman, D., Rosenfeld, R. (1998). Evaluation metrics for language models. In: Proceedings of DARPA broadcast news transcription and understanding workshop, pp. 275–280.
Chen, S. & Goodman, J. (1998). An empirical study of smoothing techniques for language modeling (Tech. Rep. No. TR-10-98).
Church, K. W., & Gale, W. A. (1991). A comparison of the enhanced Good-Turing and deleted estimation methods for estimating probabilities of English bigrams. Computer Speech & Language, 5(1), 19–54.
Dahl, E., Yu, D., Deng, L., & Acero, A. (2012). Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE Transactions on Audio, Speech and Language Processing, 20(1), 30–42.
Ganapathiraju, M., & Levin, L. (2006). TelMore: Morphological Generator for Telugu Nouns and verbs. In: Proceedings of the second international conference on universal digital library, Alexandria, Egypt, November 17–19, 2006, pp. 1–7.
Hsu, B.-J. (2007). Generalized linear interpolation of language models. In: 2007 IEEE workshop on automatic speech recognition & understanding (ASRU), 2007, pp. 136–140.
Jelinek, R. L. & Mercer, F. (1980). Interpolated estimation of Markov source parameters from sparse data. In: Proceedings of the workshop on pattern recognition in practice, North-Holland, Amsterdam, The Netherlands, 1980, pp. 381–397.
Jyothi, P., & Hasegawa-Johnson, M. (2015). Improved Hindi broadcast ASR by adapting the language model and pronunciation model using a priori syntactic and morphophonemic knowledge. In: Proceedings of the annual conference of the International Speech Communication Association, INTERSPEECH, 2015, vol. 2015-January (pp. 3164–3168).
Kalika Bali, A. R., Talukdar, P. P., & Sridhar Krishna, N. (2004). Tools for the development of a Hindi speech synthesis system. In: 5th ISCA Speech Synthesis Workshop, 2004, pp. 109–114.
Katz, S. M. (1987). Estimation of probabilities from sparse data for the language model component of a speech recognizer. IEEE Transaction on Acoustics, 35(3), 400–401.
Kneser, R. & Ney, H. (2002). Improved backing-off for M-gram language modeling. In: 1995 international conference on acoustics, speech, and signal processing, 2002, vol. 1, pp. 181–184.
Krishnamurti, B. (2009). Telugu verbal bases. Delhi: Motilal Banarsidass Publishers Pvt. Limited.
Krishnamurti, B., & Gwynn, J. P. L. (1985). A grammar of modern Telugu. Oxford: Oxford University Press.
Ney, H., Essen, U., & Kneser, R. (1994). On structuring probabilistic dependences in stochastic language modelling. Computer Speech & Language, 8(1), 1–38.
Ney, H., Martin, S., & Wessel, F. (1997). Statistical language modeling using leaving-one-out. In B. G. Spee & S. Young (Eds.), Corpus-based methods in language and speech processing (pp. 174–207). Berlin: Springer.
Pandey, A. K. & Siddiqui, T. J. (2008). An unsupervised Hindi stemmer with heuristic improvements. In: Proceedings of the second workshop on Analytics for noisy unstructured text data - AND’08, 2008, pp. 99–105.
Povey, D., et al. (2011a). The subspace Gaussian mixture model: A structured model for speech recognition. Computer Speech & Language, 25(2), 404–439.
Povey, D., et al. (2011b). The Kaldi Speech Recognition Toolkit. Okinawa: Proceedings of ASRU.
Powers, D. M. W. (1998). Applications and explanations of Zipf’s law. In: Proceedings of the joint conferences on new methods in language processing and computational natural language learning, 1998 (pp. 151–160).
Purushottam, B., Srihari Shastri, R., & Venkata Rama, D. (1978). Vyakarana padakosamu Sastra nighantuvu. Hyderabad: Telugu Acadamy.
Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of IEEE, 77(2), 257–286.
Siva Kumar, A. P., Premchand, P., & Govardhan, A. (2011). TelStem: An unsupervised telugu stemmer with heuristic improvements and normalized signatures, 2011.
Stolcke, A. (2002). SRILM an extensible language modeling toolkit. ICSLP, 2002, 901–904.
Timothy, H. W., Bell, C., & Cleary, J. G. (1990). Text compression. Upper Saddle River, NJ: Prentice-Hall, Inc.
Vegesna, V. V. R., Gurugubelli, K., Vydana, H. K., Pulugandla, B., Shrivastava, M., & Vuppala, A. K. (2017). DNN-HMM acoustic modeling for large vocabulary Telugu speech recognition. In P. R. Ghosh & R. Pal (Eds.), Mining intelligence and knowledge exploration. MIKE 2017., Lecture No. (pp. 189–197). Cham: Springer.
Venkatawdhani, D. (1976). Telugu in thirty days. Hyderabad: Andhrapradesh Sahitya Acadamy.
Witten, I. H., & Bell, T. C. (1991). The zero-frequency problem: estimating the probabilities of novel events in adaptive text compression. IEEE Transactions on Information Theory, 37(4), 1085–1094.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Pala, M., Parayitam, L. & Appala, V. Unsupervised stemmed text corpus for language modeling and transcription of Telugu broadcast news. Int J Speech Technol 23, 695–704 (2020). https://doi.org/10.1007/s10772-020-09749-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10772-020-09749-0