Abstract
This paper addresses using novel class-based language models on parallel corpora, focusing specifically on English and Chinese languages. We find that the perplexity of Chinese is generally much higher than English and discuss the possible reasons. We demonstrate the relative effectiveness of using class-based models over the modified Kneser-Ney trigram model for our task. We also introduce a rare events clustering and a polynomial discounting mechanism, which is shown to improve results. Our experimental results on parallel corpora indicate that the improvement due to classes are similar for English and Chinese. This suggests that class-based language models should be used for both languages.
Junfei Guo acknowledge support by Chinese Scholarship Council during the first author’s study in University of Stuttgart.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Manning, C.D., Schütze, H.: Foundations of statistical natural language processing. MIT Press, Cambridge (1999)
Koehn, P.: Statistical Machine Translation, 1st edn. Cambridge University Press, New York (2010)
Schütze, H.: Integrating history-length interpolation and classes in language modeling. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 1516–1525. Association for Computational Linguistics, Portland (2011)
Gao, J., Goodman, J., Li, M., Lee, K.F.: Toward a unified approach to statistical language modeling for chinese, vol. 1(1), pp. 3–33 (March 2002)
Manning, C.D., Raghavan, P., Schtze, H.: Introduction to Information Retrieval. Cambridge University Press, New York (2008)
Chang, P.C., Galley, M., Manning, C.D.: Optimizing chinese word segmentation for machine translation performance. In: Proceedings of the Third Workshop on Statistical Machine Translation, StatMT 2008, pp. 224–232. Association for Computational Linguistics, Stroudsburg (2008)
Chen, S.F., Goodman, J.: An empirical study of smoothing techniques for language modeling. In: Proceedings of the 34th Annual Meeting on Association for Computational Linguistics, ACL 1996, pp. 310–318. Association for Computational Linguistics, Stroudsburg (1996)
Brown, P.F., deSouza, P.V., Mercer, R.L., Pietra, V.J.D., Lai, J.C.: Class-based n-gram models of natural language. Computational Linguistics 18, 467–479 (1992)
Dupont, P., Rosenfeld, R.: Lattice based language models. Technical report (1997)
Eisele, A., Chen, Y.: Multiun: A multilingual corpus from united nation documents. In: Chair, N.C.C., Choukri, K., Maegaard, B., Mariani, J., Odijk, J., Piperidis, S., Rosner, M., Tapias, D. (eds.) Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC 2010). European Language Resources Association (ELRA), Valletta (2010)
Stolcke, A.: Srilm - an extensible language modeling toolkit. In: Hansen, J.H.L., Pellom, B.L. (eds.) INTERSPEECH. ISCA (2002)
Luo, X., Roukos, S.: An iterative algorithm to build chinese language models. In: Proceedings of the 34th Annual Meeting on Association for Computational Linguistics, ACL 1996, pp. 139–143. Association for Computational Linguistics, Stroudsburg (1996)
Gao, J., Goodman, J.T., Miao, J.: The Use of Clustering Techniques for Language Modeling Application to Asian Languages
Luo, J., Lamel, L., Gauvain, J.L.: Modeling characters versuswords for mandarin speech recognition. In: Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2009, pp. 4325–4328. IEEE Computer Society, Washington, DC (2009)
Schütze, H.: Distributional part-of-speech tagging. In: Proceedings of the Seventh Conference on European Chapter of the Association for Computational Linguistics, EACL 1995, pp. 141–148. Morgan Kaufmann Publishers Inc., San Francisco (1995)
Yokoyama, T., Shinozaki, T., Iwano, K., Furui, S.: Unsupervised class-based language model adaptation for spontaneous speech recognition. In: Proceedings of 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2003), vol. 1, pp. I-236–I-239 (April 2003)
Momtazi, S., Klakow, D.: A word clustering approach for language model-based sentence retrieval in question answering systems. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management, CIKM 2009, pp. 1911–1914. ACM, New York (2009)
Maltese, G., Bravetti, P., Crépy, H., Grainger, B.J., Herzog, M., Palou, F.: Combining word- and class-based language models: a comparative study in several languages using automatic and manual word-clustering techniques. In: Dalsgaard, P., Lindberg, B., Benner, H., Tan, Z.H. (eds.) INTERSPEECH, pp. 21–24. ISCA (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Guo, J., Liu, J., Walsh, M., Schmid, H. (2013). Class-Based Language Models for Chinese-English Parallel Corpus. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2013. Lecture Notes in Computer Science, vol 7817. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37256-8_22
Download citation
DOI: https://doi.org/10.1007/978-3-642-37256-8_22
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-37255-1
Online ISBN: 978-3-642-37256-8
eBook Packages: Computer ScienceComputer Science (R0)