Skip to main content
SpringerLink
Log in

A unified approach for effectively integrating source-side syntactic reordering rules into phrase-based translation

  • Original Paper
  • Published:
Language Resources and Evaluation Aims and scope Submit manuscript

Abstract

Phrase-based translation models, with sequences of words (phrases) as translation units, achieve state-of-the-art translation performance. However, phrase reordering is a major challenge for this model. Recently, researchers have focused on utilizing syntax to improve phrase reordering. In adding syntactic knowledge into phrase reordering model, using handcrafted or probabilistic syntactic rules to reorder the source-language approximating the target-language word order has been successful in improving translation quality. However, it suffers from propagating the pre-ordering errors to the later translation step (e.g. decoding). In this paper, we propose a novel framework to uniformly represent the handcrafted and probabilistic syntactic rules and integrate them more effectively into phrase-based translation. In the translation phase, for a source sentence to be translated, handcrafted or probabilistic syntactic rules are first acquired from the source parse tree prior to translation, and then instead of reordering the source sentence directly, we input these rules into the decoder and design a new algorithm to apply these rules during decoding. In order to attach more importance to the syntactic rules and distinguish reordering between syntactic and non-syntactic unit reordering, we propose to design respectively a syntactic reordering model and a non-syntactic reordering model. The syntactic rules will guide phrase reordering in decoding within the syntactic reordering model. Extensive experiments on Chinese-to-English translation show that our approach, whether incorporating handcrafted or probabilistic syntactic rules, significantly outperforms the previous methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. In SMT, phrase just denotes a sequence of words rather than a syntactic constituent. When we need to represent a syntactic constituent, we use the term “syntactic phrase”.

  2. The handcrafted rule for this case looks like NP(DNP(PP)◇NP) → NP(◇NP DNP(PP)) and will be detailed in Sect. 3.1.

  3. ◇ denotes a placeholder which indicates other syntactic nodes, in this example between PP and VP.

  4. In our proposed model, we suppose that the combination of sibling children nodes under a parent node corresponds to a syntactic phrase. Thus, the span (k + 1, j) corresponds to a syntactic phrase.

  5. In principle, MEBTG can deal with any kind of reordering. However, the reordering power is limited due to the exclusive use of lexicalized features in MEBTG.

  6. In training, the best reordered source sentence is found to be sufficient. In decoding, following (Li et al. 2007), 10-best reordered test sentences are employed as input.

  7. The catalogs include: LDC2003E14, LDC2005T06, LDC2004T07.

  8. The precision of this parser in Chinese was reported to be 78.8 in F1-value (Levy and Manning 2003).

  9. It should be noted that the handcrafted rules are extracted only on three kinds of tree nodes (VP, NP, LCP) while the probabilistic rules can be extracted on any tree node with two children beside on the tree node of VP and NP. Therefore, the probabilistic rules are much more than handcrafted rules. For pre-ordering methods, MEBTG+HSR averagely used 4.18 handcrafted rules whereas MEBTG+PSR averagely used 6.26 probabilistic rules (with probability more than 0.5) per test sentence before decoding.

  10. A lexical rule is a translation equivalent in the form of “source language phrase ||| target language phrase” in the phrase table and can be viewed as A → (x, y).

  11. http://www.icip.org.cn/cwmt2009.

  12. SBP stands for Strictly Brevity Penalty. Since the CWMT2009 workshop scores all the results with BLEU-SBP, we tune and test our system with BLEU-SBP.

References

  • Andreas, J., Habash, N., & Rambow, O. (2011). Fuzzy syntactic reordering for phrase-based statistical machine translation. In Proceedings of the 6th workshop on statistical machine translation, Edinburgh, Scotland, UK, July 30th–31th, 2011.

  • Badr, I., Zbib, R., & Glass, J. (2009). Syntactic phrase reordering for English-to-Arabic statistical machine translation. In Proceedings of the 12th conference of the European chapter of the association for computational linguistics (pp. 86–93). Athens, Greece, March 30th–April 3rd, 2009.

  • Brown, P. F., Cocke, J., Della, S. A., Pietra, V. J., Pietra, D., Jelinek, F., et al. (1990). A statistical approach to machine translation. Computational Linguistics, 16(2), 79–85.

    Google Scholar 

  • Brown, P. F., Della, S. A., Pietra, V. J., Pietra, D., & Mercer, R. L. (1993). The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2), 263–311.

    Google Scholar 

  • Cherry, C. (2008). Cohesive phrase-based decoding for statistical machine translation. In Proceedings of the 46th annual meeting of the association for computational linguistics: Human language technology (pp. 72–80). Columbus, Ohio, USA, June 15th–20th, 2008.

  • Chiang, D. (2007). Hierarchical phrase-based translation. Computational Linguistics, 33(2), 201–228.

    Article  Google Scholar 

  • Chiang, D., Marton, Y., & Resnik, P. (2008). Online large-margin training of syntactic and structural translation features. In Proceedings of the 2008 conference on empirical methods in natural language processing (pp. 224–233). Waikiki, Honolulu, USA, October 25th–27th, 2008.

  • Collins, M., Koehn, P., & Kučerová, I. (2005). Clause restructuring for statistical machine translation. In Proceedings of the 43rd annual meeting on association for computational linguistics (pp. 531–540). Michigan, USA, June 26th–30th, 2005.

  • Costa-jussà, M. R., Crego, J. M., Lambert, P., Khalilov, M., Fonollosa, J. A. R., Marino, J. B., et al. (2007). Ngram-based statistical machine translation enhanced with multiple weighted reordering hypotheses. In Proceedings of the second workshop on statistical machine translation (pp. 167–170). Prague, Czech Republic, June 27th–30th, 2007.

  • Crego, J. M., & Yvon, F. (2010). Improving reordering with linguistically informed bilingual n-grams. In Proceedings of the 23rd international conference on computational linguistics (pp. 197–205). Beijing, China, August 23rd–27th, 2010.

  • Du, J. & Way, A. (2010). The impact of source-side syntactic reordering on hierarchical phrase-based SMT. In Proceedings of the 14th annual conference of the European association for machine translation (pp. 82–89). Saint-Raphaël, France, May 27th–28th, 2010.

  • Elming, J. (2008). Syntactic reordering integrated with phrase-based SMT. In Proceedings of the 22nd international conference on computational linguistics (pp. 209–216). Manchester, UK, August 18th–22nd, 2008.

  • Galley, M., & Manning, C. D. (2009). Quadratic-time dependency parsing for machine translation. In Proceedings of the joint conference of the 47th annual meeting of the association for computational linguistics and the 4th international joint conference on natural language processing (pp. 773–781). Singapore, August 2nd–7th 2009.

  • Genzel, D. (2010). Automatically learning source-side reordering rules for large scale machine translation. In Proceedings of the 23rd international conference on computational linguistics (pp. 376–384). Beijing, China, August 23rd–27th, 2010.

  • Habash, N. (2007). Syntactic preprocessing for statistical machine translation. In Proceedings of the 11th machine translation summit (pp. 215–222). Copenhagen, Denmark, September 10th–14th, 2007.

  • Huang, L. & Chiang, D. (2007). Forest rescoring: Faster decoding with integrated language models. In Proceedings of the 45th annual meeting of the association of computational linguistics (pp. 144–151). Prague, Czech Republic, June 27th–30th, 2007.

  • Klein, D., & Manning, C. D. (2003). Accurate unlexicalized parsing. In Proceedings of the 41st annual meeting on association for computational linguistics (pp. 423–430). Sapporo, Japan, July 7th–12th, 2003.

  • Koehn, P. (2004). Statistical significance tests for machine translation evaluation. In Proceedings of the 2004 conference on empirical methods in natural language processing (pp. 388–395). Barcelona, Spain, July 25th–26th, 2004.

  • Koehn, P., Hoang, H., Birch, A., Federico, M., Bertoldi, N., Cowan, B., et al. (2007) Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th annual meeting on association for computational linguistics on interactive poster and demonstration sessions (pp. 177–180). Prague, Czech Republic, June 27th–30th, 2007.

  • Koehn, P., Och, F. J., & Marcu, D. (2003). Statistical phrase-based translation. In Proceedings of the 2003 conference of the north american chapter of the association for computational linguistics on human language (pp. 48–54). Edmonton, Canada, May 27th–June 1st, 2003.

  • Lee, Y.-S., Zhao, B., & Luo, X. (2010). Constituent reordering and syntax models for English-to-Japanese statistical machine translation. In Proceedings of the 23rd international conference on computational linguistics (pp. 626–634). Beijing, China, August 23rd–27th, 2010.

  • Levy, R., & Manning, C. D. (2003). Is it harder to parse Chinese, or the Chinese Treebank? In Proceedings of the 41st annual meeting of the association of computational linguistics (pp. 439–446).

  • Li, C.-H., Zhang, D., Li, M., Zhou, M., Li, M., & Guan, Y. (2007). A probabilistic approach to syntax-based reordering for statistical machine translation. In Proceedingd of the 45th annual meeting of the association of computational linguistics (pp. 720–727). Prague, Czech Republic, June 27th–30th, 2007.

  • Marton, Y., & Resnik, P. (2008). Soft syntactic constraints for hierarchical phrased-based translation. In Proceedings of the 46th annual meeting of the association for computational linguistics: human language technology (pp. 1003–1011), Columbus, Ohio, USA, June 15th–20th, 2008.

  • Och, F. J. (2003). Minimum error rate training in statistical machine translation. In Proceedings of the 41st annual meeting on association for computational linguistics (pp. 160–167). Sapporo, Japan, July 7th–12th, 2003.

  • Och, F. J., & Ney, H. (2003). A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1), 19–51.

    Article  Google Scholar 

  • Och, F. J., & Ney, H. (2004). The alignment template approach to statistical machine translation. Computational Linguistics, 30(4), 417–449.

    Article  Google Scholar 

  • Stolcke, A. (2002). SRILM-an extensible language modeling toolkit. In Proceedings 7th International conference on spoken language processing (pp. 901–904). Denver, Colorado, USA, September 16th–20th, 2002.

  • Tillmann, C., & Zhang, T. (2005). A localized prediction model for statistical machine translation. In Proceedings of the 43rd annual meeting on association for computational linguistics (pp. 557–564). Michigan, USA, June 26th–30th, 2005.

  • Visweswariah, K., Navratil, J., Sorensen, J., Chenthamarakshan, V., & Kambhatla, N. (2010). Syntax-based reordering with automatically derived rules for improved statistical machine translation. In Proceedings of the 23rd international conference on computational linguistics (pp. 1119–1127) Beijing, China, August 23rd–27th, 2010.

  • Wang, C., Collins, M., & Koehn, P. (2007). Chinese syntactic reordering for statistical machine translation. In Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (pp. 737–745). Prague, Czech Republic, June 27th–30th, 2007.

  • Wu, D. (1997). Stochastic inversion transduction grammars and bilingual parsing of parallel corpora. Computational Linguistics, 23(3), 377–403.

    Google Scholar 

  • Wu, X., Sudoh, K., Duh, K., Tsukada, H., & Nagata, M. (2011). Extracting pre-ordering rules from predicate-argument structures. In Proceedings of the 5th international joint conference on natural language processing (pp. 29–37). Chiang Mai, Thailand, November 8th–13th, 2011.

  • Xiang, B., Ge, N., & Ittycheriah, A. (2011). Improving reordering for statistical machine translation with smoothed priors and syntactic features. In Proceedings of the fifth workshop on syntax, semantics and structure in statistical translation (pp. 61–69). Portland, Oregon, USA, June 19th–24th, 2011.

  • Xiong, D., Liu, Q., & Lin, S. (2006). Maximum entropy based phrase reordering model for statistical machine translation. In Proceedings of the 21st international conference on computational linguistics and the 44th annual meeting of the association for computational linguistics (pp. 521–528). Sydney, Australia, July 17th–21st, 2006.

  • Xiong, D., Zhang, M., Aw, A., & Li, H. (2008). Linguistically annotated BTG for statistical machine translation. In Proceedings of the 22nd international conference on computational linguistics (pp. 1009–1016). Manchester, UK, August 18th–22nd, 2008.

  • Xiong, D., Zhang, M., & Li, H. (2011). Enhancing language models in statistical machine translation with backward N-grams and mutual information triggers. In Proceedings of the 49th annual meeting of the association for computational linguistics (pp. 1288–1297). Portland, Oregon, USA, June 19th–24th, 2011.

  • Xu, P., Kang, J., Ringgaard, M., & Och, F. (2009). Using a dependency parser to improve SMT for subject-object-verb languages. In Proceedings of human language technologies: The 2009 annual conference of the North American chapter of the association for computational linguistics (pp. 245–253). Boulder Colorado, May 31th–June 5th, 2009.

  • Xue, N., Xia, F., Chiou, F.-D., & Palmer, M. (2005). The Penn Chinese Treebank: Phrase structure annotation of a large corpus. Natural Language Engineering, 11(02), 207–238.

    Article  Google Scholar 

  • Younger, D. H. (1967). Recognition and parsing of context-free languages in time n3. Information and Control, 10(2), 189–208.

    Article  Google Scholar 

  • Zens, R., Ney, H., Watanabe, T., & Sumita, E. (2004). Reordering constraints for phrase-based statistical machine translation. In Proceedings of the 20th international conference on computational linguistics (pp. 205–262). Geneva, Switzerland, August 23rd–27th, 2004.

  • Zhang, L. (2004). Maximum entropy modeling toolkit for Python and C++. Available at http://homepages.inf.ed.ac.uk/s0450736/maxent_toolkit.html.

  • Zhang, M., & Li, H. (2009). Tree kernel-based SVM with structured syntactic knowledge for BTG-based phrase reordering. In Proceedings of the 2009 conference on empirical methods in natural language processing (pp. 698–707). Singapore, August 6th–7th, 2009.

  • Zhang, D., Li, M., Li, C.-H., & Zhou, M. (2007). Phrase reordering model integrating syntactic knowledge for SMT. In Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (pp. 533–540) Prague, Czech Republic, June 27th–30th, 2007.

  • Zollmann, A., & Venugopal, A. (2006). Syntax augmented machine translation via chart parsing. In Proceedings of NAACL 2006—Workshop on statistical machine translation. New York. June 4–9.

Download references

Author information

Authors and Affiliations

  1. National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Jiajun Zhang & Chengqing Zong

Authors
  1. Jiajun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chengqing Zong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiajun Zhang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhang, J., Zong, C. A unified approach for effectively integrating source-side syntactic reordering rules into phrase-based translation. Lang Resources & Evaluation 47, 449–474 (2013). https://doi.org/10.1007/s10579-013-9217-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10579-013-9217-4

Keywords

Search

Navigation