Abstract
This paper evaluates a machine translation (MT) system based on the interlingua approach, the Universal Network Language (UNL) system, designed for Multilanguage translation. The study addresses evaluation of English-Arabic translation and aims at comparing the MT systems based on UNL against other systems. Also, it serves to analyze the development of the system understudy by comparing output at the sentence level. The evaluation is performed on the Encyclopedia of Life Support Systems (EOLSS), a wide range corpus covering multiple linguistic and cultural backgrounds. Three automated metrics are evaluated, namely BLEU, F1 and Fmean after being adapted to the Arabic language. Results revealed that the UNL MT outperforms other systems for all metrics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Agrawal, A., Lavie, A.: METEOR, M-BLEU and M-TER: Evaluation Metrics For High Correlation with Human Rankings of Machine Translation Output. In: Proc. of the 3rd Workshop on Statistical Machine Translation, Ohio, June 2008, pp. 115–118 (2008)
Alansary, S., Nagi, M., Adly, N.: A Semantic Based Approach for Multilingual Translation of Massive Documents. In: The 7th International Symposium on Natural Language Processing (SNLP), Pattaya, Thailand (2007)
Alansary, S., Nagi, M., Adly, N.: Generating Arabic text: The Decoding Component of an Interlingual System for Man-Machine Communication in Natural Language. In: The 6th International Conference on Language Engineering, Cairo, Egypt, December 6-7 (2006)
Banerjee, S., Lavie, A.: METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In: Proc. of ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and Summarization, Ann Arbor (2005)
Cormen, T., Leiserson, C., Rivest, R., Stein, C.: Introduction to Algorithms, 2nd edn. MIT Press, Cambridge (2001)
Encyclopedia of Life Support Systems, http://www.eolss.net/
Lavie, A., Agarwal, A.: METEOR: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments. In: Proceedings of the Second ACL Workshop on Statistical Machine Translation, Prague, June 2007, pp. 228–231 (2007)
Lavie, A., Pianesi, F., Levin, L.: The NESPOLE! System for Multilingual Speech Communication over the Internet. IEEE Transactions on Audio, Speech and Language Processing 14(5) (September 2006)
Lavie, A., Sagae, K., Jayaraman, S.: The Significance of Recall in Automatic Metrics for MT Evaluation. In: Frederking, R.E., Taylor, K.B. (eds.) AMTA 2004. LNCS (LNAI), vol. 3265, pp. 134–143. Springer, Heidelberg (2004)
Lee, Y., Yi, W., Seneff, S., Weinstein, C.: Interlingua-based Broad-coverage Korean-to-English Translation in CCLINC. In: Proceedings of the 1st International Conference on Human Language Technology Research, san Diego (2001)
Leusch, G., Ueffing, N., Ney, H.: CDER: Efficient MT Evaluation Using Block Movements. In: Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics (2006)
Lin, C., Och, J.: ORANGE: a Method for Evaluation Automatic Evaluation Metrics for Machine Translation. In: COLING 2004, Switzerland, August 2004, pp. 501–507 (2004)
Lopez, A.: Statistical Machine Translation. In: ACM Comp. Surveys, vol. 40 (August 2008)
Melamed, I., Green, R., Turian, J.: Precision and Recall of Machine Translation. In: Proceedings of the HLTNAACL 2003, Canada, pp. 61–63 (2003)
Papineni, K., Roukos, S., Ward, T., Zhu, W.: BLEU: a Method for Automatic Evaluation of Machine Translation. In: 40th Annual Meeting of the Association for Computational Linguistics (ACL), Philadelphia, pp. 311–318 (2002)
Papineni, K., Roukos, S., Ward, T., Henderson, J., Reeder, F.: Corpus-based Comprehensive and diagnostic MT evaluation: Initial Arabic, Chinese, French, and Spanish results. In: Proceedings of Human Language Technology 2002, San Diego, CA (2002)
Przybocki, M., Sanders, G., Le, A.: Edit Distance: a Metric for Machine Translation Evaluation. In: LREC (2006)
Rijsbergen, C.: Information Retrieval, 2nd edn., Butterworths, London (1979)
Shaalan, K., Monem, A., Rafea, A., Baraka, H.: Mapping Interlingua Representations to Feature Structures of Arabic Sentences. In: The Challenge of Arabic for NLP/MT. International Conference at the British Computer Society, London, pp. 149–159 (2006)
Sinha, R., et al.: ANGLABHARTI: a Multilingual Machine Aided Translation from English to Indian Languages. In: IEEE Intl. Conference on Systems, Man and Cybernetics (1995)
Snover, M., Dorr, B., Schwartz, R., Micciulla, L.: A Study of Translation Edit Rate with Targeted Human Annotation. In: Proceedings of AMTA, Boston (2006)
Turian, J., Shen, L., Melamed, I.: Evaluation of Machine Translation and its Evaluation. In: Proceedings of MT Summit IX (2003)
Uchida, H., Zhu, M., Della Senta, T.: The Universal Networking Language. In: UNDL Foundation (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Adly, N., Al Ansary, S. (2010). Evaluation of Arabic Machine Translation System Based on the Universal Networking Language. In: Horacek, H., Métais, E., Muñoz, R., Wolska, M. (eds) Natural Language Processing and Information Systems. NLDB 2009. Lecture Notes in Computer Science, vol 5723. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12550-8_20
Download citation
DOI: https://doi.org/10.1007/978-3-642-12550-8_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12549-2
Online ISBN: 978-3-642-12550-8
eBook Packages: Computer ScienceComputer Science (R0)