Abstract
Constructing a classifier that distinguishes machine translations from human translations is a promising approach to automatic evaluation of machine-translated sentences. Using this approach, we constructed a classifier using Support Vector Machines based on word-alignment distributions between source sentences and human or machine translations. This paper investigates the validity of the classification-based method by comparing it with well-known evaluation methods. The experimental results show that our classification-based method can accurately evaluate fluency of machine translations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Corston-Oliver, S., Gamon, M., Brockett, C.: A Machine Learning Approach to the Automatic Evaluation of Machine Translation. In: Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics, Toulouse, France, pp. 148–155 (2001)
Kulesza, A., Shieber, S.M.: A Learning Approach to Improving Sentence-level MT Evaluation. In: Proceedings of the 10th International Conference on Theoretical and Methodological Issues in Machine Translation, Baltimore, Maryland, pp. 75–84 (2004)
Kotani, K., Yoshimi, T., Kutsumi, T., Sata, I., Isahara, H.: A Classification Approach to Automatic Evaluation of Machine Translation Based on Word Alignment. Language Forum 34, 153–168 (2008)
Kotani, K., Yoshimi, T., Kutsumi, T., Sata, I.: Validity of an Automatic Evaluation of Machine Translation Using a Word-Alignment-Based Classifier. In: Li, W., Mollá-Aliod, D. (eds.) ICCPOL 2009. LNCS(LNAI), vol. 5459, pp. 91–102. Springer, Heidelberg (2009)
Kotani, K., Yoshimi, T., Kutsumi, T., Sata, I., Isahara, H.: A Method of Automatically Evaluating Machine Translations Using a Word-alignment-based Classifier. In: Proceedings of the Workshop “Mixing Approaches to Machine Translation” (MATMT), pp. 11–18 (2008)
Papineni, K.A., Roukos, S., Ward, T., Zhu, W.-J.: Bleu: A Method for Automatic Evaluation of Machine Translation. Technical Report RC22176. IBM Research Division, Thomas J. Watson Research Center (2001)
Doddington, G.: Automatic Evaluation of Machine Translation Quality Using N-gram Co-occurrence Statistics. In: Proceedings of the 2nd Human Language Technology Conference, San Diego, California, pp. 128–132 (2002)
Banerjee, S., Alon, L.: METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In: Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, Ann Arbor, Michigan, pp. 65–72 (2005)
Quinlan, J.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1992)
Vapnik, V.: Statistical Learning Theory. Wiley Interscience, New York (1998)
Och, F.J., Ney, H.: A Systematic Comparison of Various Statistical Alignment Models. Computational Linguistics 29(1), 19–51 (2003)
Utiyama, M., Isahara, H.: Reliable Measures for Aligning Japanese-English News Articles and Sentences. In: Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, Sapporo, Japan, pp. 72–79 (2003)
Sharp, Hon’yaku Kore Ippom (2003)
Fujitsu, Atlas Personal Translation (2005)
LogoVista, LogoVista X PRO ver.3.0 (2004)
Chasen, http://chasen-legacy.sourceforge.jp/ (in Japanese)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kotani, K., Yoshimi, T. (2010). A Machine Learning-Based Evaluation Method for Machine Translation. In: Konstantopoulos, S., Perantonis, S., Karkaletsis, V., Spyropoulos, C.D., Vouros, G. (eds) Artificial Intelligence: Theories, Models and Applications. SETN 2010. Lecture Notes in Computer Science(), vol 6040. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12842-4_42
Download citation
DOI: https://doi.org/10.1007/978-3-642-12842-4_42
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12841-7
Online ISBN: 978-3-642-12842-4
eBook Packages: Computer ScienceComputer Science (R0)