Skip to main content
Log in

Abstract

Analyzing the syntactic structure of natural languages by parsing is an important task in artificial intelligence. Due to the complexity of natural languages, individual parsers tend to make different yet complementary errors. We propose a neural network based approach to combine parses from different parsers to yield a more accurate parse than individual ones. Unlike conventional approaches, our method directly transforms linearized candidate parses into the ground-truth parse. Experiments on the Penn English Treebank show that the proposed method improves over a state-of-the-art parser combination approach significantly.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Collins M. Three generative, lexicalised models for statistical parsing. In Proc. ACL, July 1997, pp.16-23.

  2. Charniak E. A maximum-entropy-inspired parser. In Proc. NAACL, April 29-May 4, 2000, pp.132-139.

  3. Klein D, Manning C D. Accurate unlexicalized parsing. In Proc. ACL, July 2003, pp.423-430.

  4. McDonald R, Pereira F, Ribarov K, Hajič J. Non-projective dependency parsing using spanning tree algorithms. In Proc. EMNLP, October 2005, pp.523-530.

  5. Petrov S, Barrett L, Thibaux R, Klein D. Learning accurate, compact, and interpretable tree annotation. In Proc. ACL, July 2006, pp.433-440.

  6. Nivre J. Incremental non-projective dependency parsing. In Proc. HLT-NAACL, April 2007, pp.396-403.

  7. Huang L, Sagae K. Dynamic programming for linear-time incremental parsing. In Proc. ACL, July 2010, pp.1077-1086.

  8. Zhang Y, Clark S. Syntactic processing using the generalized perceptron and beam search. Computational Linguistics, 2011, 37(1): 105-151.

    Article  Google Scholar 

  9. Socher R, Lin C, Ng A Y, Manning C D. Parsing natural scenes and natural language with recursive neural networks. In Proc. ICML, June 28-July 2, 2011, pp.129-136.

  10. Chen D, Manning C D. A fast and accurate dependency parser using neural networks. In Proc. EMNLP, October 2014, pp.740-750.

  11. Dyer C, Ballesteros M, Ling W, Matthews A, Smith N A. Transition-based dependency parisng with stack long shortmemory. In Proc. ACL, July 2015, pp.334-343.

  12. Vinyals O, Kaiser L, Koo T, Petrov S, Sutskever I, Hinton G. Grammar as a foreign language. In Proc. NIPS, December 2015, pp.2773-2781.

  13. Weiss D, Alberti C, Collins M, Petrov S. Structured training for neural network transition-based parsing. In Proc. ACL, July 2015, pp.323-333.

  14. Andor D, Alberti C, Weiss D, Severyn A, Presta A, Ganchev K, Petrov S, Collins M. Globally normalized transition-based neural networks. In Proc. ACL, August 2016, pp.2442-2452.

  15. Dyer C, Kuncoro A, Ballesteros M, Smith N A. Recurrent neural network grammars. In Proc. HLT-NAACL, June 2016, pp.199-209.

  16. Kiperwasser E, Goldberg Y. Simple and accurate dependency parsing using bidirectional LSTM feature representations. Transactions of the Association for Computational Linguistics, 2016, 4: 313-327.

    Google Scholar 

  17. Henderson J C, Brill E. Exploiting diversity in natural language processing. In Proc. EMNLP, June 1999, pp.187-194.

  18. Charniak E. Statistical parsing with a context-free grammar and word statistics. In Proc. AAAI, July 1997, pp.598-603.

  19. McDonald R, Nivre J. Analyzing and integrating dependency parsers. Computational Linguistics, 2011, 37(1): 197-230.

    Article  Google Scholar 

  20. Sagae K, Lavie A. Parser combination by reparsing. In Proc. HLT-NAACL, June 2006, pp.129-132.

  21. Fossum V, Knight K. Combining constituent parsers. In Proc. HLT-NAACL, May 31-June 5, 2009, pp.253-256.

  22. Zhang H, Zhang M, Tan C L, Li H. K-best combination of syntactic parsers. In Proc. EMNLP, August 2009, pp.1552-1560.

  23. Johnson M, Ural A E. Reranking the Berkeley and Brown parsers. In Proc. HLT-NAACL, June 2010, pp.665-668.

  24. Narayan S, Cohen S B. Diversity in spectral learning for natural language parsing. In Proc. EMNLP, September 2015, pp.1868-1878.

  25. Choe D K, McClosky D, Charniark E. Syntactic parse fusion. In Proc. EMNLP, September 2015, pp.1360-1366.

  26. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation, 1997, 9(8): 1735-1780.

    Article  Google Scholar 

  27. Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. In Proc. ICLR, May 2015.

  28. Zoph B, Knight K. Multi-source neural translation. In Proc. HLT-NAACL, June 2016, pp.30-34.

  29. Firat O, Cho K, Bengio Y. Multi-way, multilingual neural machine translation with a shared attention mechanism. In Proc. HLT-NAACL, June 2016, pp.866-875.

  30. Toutanova K, Klein D,Manning C D, Singer Y. Feature-rich part-of-speech tagging with a cyclic dependency network. In Proc. HLT-NAACL, May 27-June 1, 2003, pp.173-180.

  31. Zhu M, Zhang Y, Chen W, Zhang M, Zhu J. Fast and accurate shift-reduce constituent parsing. In Proc. ACL, August 2013, pp.434-443.

  32. Collins M, Koo T. Discriminative reranking for natural language parsing. Computational Linguistics, 2005, 31(1): 25-70.

    Article  MathSciNet  MATH  Google Scholar 

  33. Kingma D, Ba J. Adam: A method for stochastic optimization. In Proc. ICLR, May 2015.

  34. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 2014, 15(1): 1929-1958.

    MathSciNet  MATH  Google Scholar 

  35. Pascanu R, Mikolov T, Bengio Y. On the difficulty of training recurrent neural networks. In Proc. ICML, June 2013, pp.1310-1318.

  36. Shen S, Cheng Y, He Z, He W, Wu H, Sun M, Liu Y. Minimum risk training for neural machine translation. In Proc. ACL, August 2016, pp.1683-1692.

Download references

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yang Liu.

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM 1

(PDF 472 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, LE., Sun, MS., Cheng, Y. et al. Neural Parse Combination. J. Comput. Sci. Technol. 32, 749–757 (2017). https://doi.org/10.1007/s11390-017-1756-5

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-017-1756-5

Keywords

Navigation