Skip to main content
SpringerLink
Log in

Evaluation of Semantic Parsing Frameworks for Automated Knowledge Base Construction

  • Conference paper
  • First Online:
Intelligent Systems Design and Applications (ISDA 2022)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 646))

  • 191 Accesses

Abstract

Semantic parsing is a subfield of natural language understanding that translates natural language utterances into detailed representations of their meaning. Though numerous meaning representation frameworks exist, none has been universally accepted. The current paper provides a comparative study of the most promising semantic representations of text, taking into account parsers, performance, extensibility, available corpora, and other aspects - used for the automated construction of a knowledge base for commonsense reasoning. In parallel, a corpus was constructed, and experiments were conducted to capture the linguistic attributes essential for automated reasoning. Based on the findings the author suggests using said parsers in an ensemble.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 229.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 299.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/jflanigan/jamr.

  2. 2.

    https://github.com/IBM/transition-amr-parser.

  3. 3.

    https://github.com/bjascob/amrlib.

  4. 4.

    https://github.com/SUDA-LA/ucca-parser.

  5. 5.

    https://github.com/danielhers/tupa.

  6. 6.

    https://universaldependencies.org/.

  7. 7.

    https://github.com/sivareddyg/UDepLambda.

  8. 8.

    https://github.com/UppsalaNLP/uuparser.

  9. 9.

    https://stanfordnlp.github.io/stanza/.

  10. 10.

    http://moin.delph-in.net/wiki/EdsTop.

  11. 11.

    https://github.com/delph-in/pydelphin.

  12. 12.

    https://github.com/draplater/hrg-parser.

  13. 13.

    https://catalog.ldc.upenn.edu/LDC2015T13.

  14. 14.

    https://github.com/ufal/perin.

  15. 15.

    http://mrp.nlpl.eu/2020/.

  16. 16.

    https://github.com/LeonCrashCode/TreeDRSparsing/tree/bs_sattn_drssup.

  17. 17.

    https://github.com/EdinburghNLP/EncDecDRSparsing.

  18. 18.

    http://decomp.io/.

  19. 19.

    https://github.com/hltcoe/PredPatt.

  20. 20.

    https://github.com/esteng/miso_uds.

  21. 21.

    https://huggingface.co/datasets/commonsense_qa.

  22. 22.

    https://www.cs.utexas.edu/users/ml/nldata/geoquery.html.

  23. 23.

    https://github.com/martinve/isda2022/blob/main/example.md.

  24. 24.

    https://github.com/sriram-c/ucca-tool.

  25. 25.

    https://universaldependencies.org/en/dep/.

References

  1. Davis, E.: Logical formalizations of commonsense reasoning: a survey. J. Artif. Intell. Res. 59, 651–723 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  2. Kamath, A., Das, R.: A survey on semantic parsing. In: Automated Knowledge Base Construction (AKBC) (2018)

    Google Scholar 

  3. Abzianidze, L., et al.: MRP 2020: the second shared task on cross framework and cross lingual meaning representation parsing, pp. 1–22. Association for Computational Linguistics (ACL) (2020)

    Google Scholar 

  4. Pavlova, S., Amblard, M., Guillaume, B.: How Much Of UCCA Can Be Predicted From AMR? In: Proceedings of the 18th Joint ACL - ISO Workshop on Interoperable Semantic Annotation within LREC2022, pp. 110–117 (2022)

    Google Scholar 

  5. van Noord, R., Abzianidze, L., Haagsma, H., Bos, J.: Evaluating scoped meaning representations. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018) (2018)

    Google Scholar 

  6. Abend, O., Rappoport, A.: UCCA: a semantics-based grammatical annotation scheme. In: IWCS, vol. 13, pp. 1–12 (2013)

    Google Scholar 

  7. Nivre, J., et al.: Universal dependencies v2: an evergrowing multilingual treebank collection. In: Proceedings of the 12th Language Resources and Evaluation Conference, pp. 4034–4043 (2020)

    Google Scholar 

  8. Haverinen, K., Nyblom, J., Viljanen, T., Laippala, V., Kohonen, S., Missilä, A., Ojala, S., Salakoski, T., Ginter, F.: Building the essential resources for Finnish the Turku dependency treebank. Lang. Resour. Eval. 48(3), 493–531 (2014)

    Article  MATH  Google Scholar 

  9. Reddy, S., Täckström, O., Petrov, S., Steedman, M., Lapata, M.: Universal Semantic Parsing. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 89–101 (2017)

    Google Scholar 

  10. Kiperwasser, E., Goldberg, Y.: Simple and accurate dependency parsing using bidirectional LSTM feature representations. Trans. Assoc. Comput. Linguist. 4, 313–327 (2016)

    Article  Google Scholar 

  11. Copestake, A., Flickinger, D., Pollard, C., Sag, I.A.: Minimal recursion semantics: an introduction. Res. Lang. Comput. 3(2), 281–332 (2005)

    Article  Google Scholar 

  12. Oepen, S., Lønning, J.T.: Discriminant-based MRS banking. In: LREC, pp. 1250–1255 (2006)

    Google Scholar 

  13. Hajic, J., et al.: Announcing Prague Czech-English Dependency Treebank 2.0. In: Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pp. 3153–3160 (2012)

    Google Scholar 

  14. Samuel, D., Straka, M.: UFAL at MRP 2020: permutation-invariant semantic parsing in PERIN. CoNLL 2020, 53 (2020)

    Google Scholar 

  15. Basile, V., Bos, J., Evang, K., Venhuizen, N.: Developing a large semantically annotated corpus. In: LREC 2012, Eighth International Conference on Language Resources and Evaluation (2012)

    Google Scholar 

  16. Liu, Y., Che, W., Zheng, B., Qin, B., Liu, T.: An AMR aligner tuned by transition-based parser. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2422–2430 (2018)

    Google Scholar 

  17. Bos, J., Basile, V., Evang, K., Venhuizen, N.J., Bjerva, J.: The groningen meaning bank. In: Ide, N., Pustejovsky, J. (eds.) Handbook of Linguistic Annotation, pp. 463–496. Springer, Dordrecht (2017). https://doi.org/10.1007/978-94-024-0881-2_18

    Chapter  Google Scholar 

  18. Liu, J., Cohen, S., Lapata, M.: Discourse representation structure parsing. In: 56th Annual Meeting of the Association for Computational Linguistics, pp. 429–439. Association for Computational Linguistics (ACL) (2018)

    Google Scholar 

  19. White, A.S., et al.: Universal decompositional semantics on universal dependencies. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 1713–1723 (2016)

    Google Scholar 

  20. White, A.S., et al.: The universal decompositional semantics dataset and decomp toolkit. In: Proceedings of the 12th Language Resources and Evaluation Conference, pp. 5698–5707 (2020)

    Google Scholar 

  21. Kollar, T., et al.: The alexa meaning representation language. In: NAACL-HLT (3), pp. 177–184 (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tallinn University of Technology, Tallinn, Estonia

    Martin Verrev

Authors
  1. Martin Verrev
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martin Verrev .

Editor information

Editors and Affiliations

  1. Faculty of Computing and Data Science, FLAME University, Pune, Maharashtra, India

    Ajith Abraham

  2. Center for Smart Computing Continuum, Burgenland, Austria

    Sabri Pllana

  3. University of Bari, Bari, Italy

    Gabriella Casalino

  4. University of Jinan, Jinan, Shandong, China

    Kun Ma

  5. Department of Computer Science and Engineering, Thapar Institute of Engineering and Technology, Patiala, Punjab, India

    Anu Bajaj

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Verrev, M. (2023). Evaluation of Semantic Parsing Frameworks for Automated Knowledge Base Construction. In: Abraham, A., Pllana, S., Casalino, G., Ma, K., Bajaj, A. (eds) Intelligent Systems Design and Applications. ISDA 2022. Lecture Notes in Networks and Systems, vol 646. Springer, Cham. https://doi.org/10.1007/978-3-031-27440-4_53

Download citation

Publish with us

Policies and ethics

Search

Navigation