Skip to main content
Springer Nature Link
Log in

Implementing Statistical Machine Translation into Mobile Augmented Reality Systems

  • Conference paper
  • First Online:
Multimedia and Network Information Systems

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 506))

  • 581 Accesses

Abstract

A statistical machine translation (SMT) capability would be very useful in augmented reality (AR) systems. For example, translating and displaying text in a smart phone camera image would be useful to a traveler needing to read signs and restaurant menus, or reading medical documents when a medical problem arises when visiting a foreign country. Such system would also be useful for foreign students to translate lectures in real time on their mobile devices. However, SMT quality has been neglected in AR systems research, which has focused on other aspects, such as image processing, optical character recognition (OCR), distributed architectures, and user interaction. In addition, general-purpose translation services, such as Google Translate, used in some AR systems are not well-tuned to produce high-quality translations in specific domains and are Internet connection dependent. This research devised SMT methods and evaluated their performance for potential use in AR systems. We give particular attention to domain-adapted SMT systems, in which an SMT capability is tuned to a particular domain of text to increase translation quality. We focus on translation between the Polish and English languages, which presents a number of challenges due to fundamental linguistic differences. However, the SMT systems used are readily extensible to other language pairs. SMT techniques are applied to two domains in translation experiments: European Medicines Agency (EMEA) medical leaflets and the Technology, Entertainment, Design (TED) lectures. In addition, field experiments are conducted on random samples of Polish text found in city signs, posters, restaurant menus, lectures on biology and computer science, and medical leaflets. Texts from these domains are translated by a number of SMT system variants, and the systems’ performance is evaluated by standard translation performance metrics and compared. The results appear very promising and encourage future applications of SMT to AR systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Carmigniani, J.: Augmented Reality Methods and Algorithms for Hearing Augmentation. Florida Atlantic University (2011)

    Google Scholar 

  2. Fragoso, V., et al.: Translatar: a mobile augmented reality translator. In: 2011 IEEE Workshop on Applications of Computer Vision (WACV), pp. 497–502. IEEE (2011)

    Google Scholar 

  3. Gschwandtner, M., Kritz, M., Boyer, C.: Requirements of the health professional research. Technical Report (2011)

    Google Scholar 

  4. NeWo LLC.: EnWo—English cam translator. https://play.google.com/store/apps/details?id=com.newo.enwo. Accessed 12 Jan 2014

  5. Cui, L., et al.: Multi-domain adaptation for SMT using multi-task learning. In: EMNLP, pp. 1055–1065 (2013)

    Google Scholar 

  6. Google.: Translate API. https://cloud.google.com/translate/v2/pricing. Accessed 6 April 2015

  7. Wołk, K., Marasek, K.: A sentence meaning based alignment method for parallel text corpora preparation. New Perspectives in Information Systems and Technologies, vol. 1, pp. 229–237. Springer International Publishing (2014)

    Google Scholar 

  8. Martedi, S., Uchiyama, H., Saito, H.: Clickable augmented documents. In: 2010 IEEE International Workshop on Multimedia Signal Processing (MMSP), pp. 162–66. IEEE (2010)

    Google Scholar 

  9. Abbyy.: TextGrabber + Translator. http://abbyy-textgrabber.android.informer.com/. Accessed 16 Feb 2015

  10. Du, J., et al.: Snap and translate using windows phone. In: 2011 International Conference on Document Analysis and Recognition (ICDAR), pp. 809–813. IEEE (2011)

    Google Scholar 

  11. The Economist. Word Lens: This changes everything. http://www.economist.com/blogs/gulliver/2010/12/instant_translation. Accessed 18 Dec 2010

  12. DATAMARK, Inc.: OCR as a real-time language translator. https://www.datamark.net/blog/ocr-as-a-real-time-language-translator. Accessed 30 July 2012

  13. Khan, T., et al.: Augmented reality based word translator. Int. J. Innov. Res. Comput. Sci. Technol. (IJIRCST) 2(2) (2014)

    Google Scholar 

  14. Mahbub-Uz-Zaman, S., Islam, T.: Application of augmented reality: mobile camera based bangla text detection and translation. 2012. PhD Thesis. BRAC University (2012)

    Google Scholar 

  15. Emmanuel, A.S., Nithyanandam, S.: An optimal text recognition and translation system for smart phones using genetic programming and cloud. Int. J. Eng. Sci. Innov. Technol. (IJESIT) 3(2), 437–443 (2014)

    Google Scholar 

  16. Toyama, T., et al.: A mixed reality head-mounted text translation system using eye gaze input. In: Proceedings of the 19th International Conference on Intelligent User Interfaces, pp. 329–334. ACM (2014)

    Google Scholar 

  17. Koehn, P., Schroeder, J.: Experiments in domain adaptation for statistical machine translation. In: Proceedings of the Second Workshop on Statistical Machine Translation. Association for Computational Linguistics, pp. 224–227 (2007)

    Google Scholar 

  18. Koehn, P., et al.: Moses: open source toolkit for statistical machine translation. In: Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions. Association for Computational Linguistics, pp. 177–180 (2007)

    Google Scholar 

  19. Wołk, K., Marasek, K.: Polish–English speech statistical machine translation systems for the IWSLT 2013. In: Proceedings of the 10th International Workshop on Spoken Language Translation, Heidelberg, Germany, pp. 113–119 (2013)

    Google Scholar 

  20. Wołk, K., Marasek, K.: A sentence meaning based alignment method for parallel text corpora preparation. New Perspectives in Information Systems and Technologies, vol. 1, pp. 229–237. Springer International Publishing (2014)

    Google Scholar 

  21. Koehn, P. What is a better translation? Reflections on six years of running evaluation campaigns. Tralogy (2011)

    Google Scholar 

  22. Wołk, K., Marasek, K.: Enhanced bilingual evaluation understudy. Lect. Notes Inf. Theory 2(2) (2014)

    Google Scholar 

  23. Banerjee, S., Lavie, A.: Meteor: an automatic metric for MT evaluation with improved correlation with human judgments. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pp. 65–72 (2005)

    Google Scholar 

  24. Snover, M., et al.: A study of translation edit rate with targeted human annotation. In: Proceedings of Association for Machine Translation in the Americas, pp. 223–231 (2006)

    Google Scholar 

  25. Smith, R.: An overview of the tesseract OCR engine. ICDAR, pp. 629–633. IEEE (2007)

    Google Scholar 

  26. Heafield, K.: KenLM: faster and smaller language model queries. In: Proceedings of the Sixth Workshop on Statistical Machine Translation. Association for Computational Linguistics, pp. 187–197 (2011)

    Google Scholar 

  27. Information Systems Laboratory, Adam Mickiewicz University.: SyMGIZA++. http://psi.amu.edu.pl/en/index.php?title=SyMGIZA. Accessed 27 April 2011

  28. Gao, Q., Vogel, S.: Parallel implementations of word alignment tool. Software Engineering, Testing, and Quality Assurance for Natural Language Processing. Association for Computational Linguistics, pp. 49–57 (2008)

    Google Scholar 

  29. Durrani, N., et al.: Integrating an unsupervised transliteration model into statistical machine translation. EACL 2014, 148 (2014)

    Google Scholar 

  30. Johnson, J.H., et al.: Improving translation quality by discarding most of the phrase table. Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL) (2007)

    Google Scholar 

  31. Lavie, A.: Evaluating the output of machine translation systems. AMTA Tutorial (2010)

    Google Scholar 

  32. KantanMT—a sophisticated and powerful machine translation solution in an easy-to-use package. http://www.kantanmt.com

Download references

Acknowledgments

This research was supported by Polish-Japanese Academy of Information Technology statutory resources (ST/MUL/2016) and resources for young researchers.

Author information

Authors and Affiliations

  1. Polish-Japanese Academy of Information Technology, ul. Koszykowa 86, 02-008, Warsaw, Poland

    Krzysztof Wołk, Agnieszka Wołk & Krzysztof Marasek

Authors
  1. Krzysztof Wołk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Agnieszka Wołk
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Krzysztof Marasek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Krzysztof Wołk or Agnieszka Wołk .

Editor information

Editors and Affiliations

  1. Department of Information Systems, Faculty of Computer Science and Management, Wrocław University of Science and Technology, Wroclaw, Poland

    Aleksander Zgrzywa

  2. Department of Information Systems, Faculty of Computer Science and Management, Wrocław University of Science and Technology, Wrocław, Poland

    Kazimierz Choroś

  3. Department of Information Systems, Faculty of Computer Science and Management, Wrocław University of Science and Technology, Wrocław, Poland

    Andrzej Siemiński

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing Switzerland

About this paper

Cite this paper

Wołk, K., Wołk, A., Marasek, K. (2017). Implementing Statistical Machine Translation into Mobile Augmented Reality Systems. In: Zgrzywa, A., Choroś, K., Siemiński, A. (eds) Multimedia and Network Information Systems. Advances in Intelligent Systems and Computing, vol 506. Springer, Cham. https://doi.org/10.1007/978-3-319-43982-2_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-43982-2_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-43981-5

  • Online ISBN: 978-3-319-43982-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics