Skip to main content
SpringerLink
Log in

Improving the Consistency of Semantic Parsing in KBQA Through Knowledge Distillation

  • Conference paper
  • First Online:
Web and Big Data (APWeb-WAIM 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14333))

  • 89 Accesses

Abstract

Knowledge base question answering (KBQA) is an important task that involves analyzing natural language questions and retrieving relevant answers from a knowledge base. To achieve this, Semantic Parsing (SP) is used to parse the question into a structured logical form, which is then executed to obtain the answer. Although different logical forms have unique advantages, existing methods only focus on a single logical form and do not consider the semantic consistency between different logical forms. In this paper, we address the issue of consistency in semantic parsing, which has not been explored before. We show that improving the semantic consistency between multiple logical forms can help increase the parsing performance. To address the consistency problem, we present a dynamic knowledge distillation framework for semantic parsing (DKD-SP). Our framework enables one logical form to learn some useful hidden knowledge from another, which improves the semantic consistency of different logical forms. Additionally, it dynamically adjusts the supervised weight of the hidden knowledge as the student model’s ability changes. We evaluate our approach on the KQA Pro dataset, and our experimental results confirm its effectiveness. Our method improves the overall accuracy of the seven types of questions by 0.57%, with notable improvements in the accuracy of Qualifier, Compare, and Count questions. Furthermore, in the compositional generalization scenario, the overall accuracy improved by 4.02%. Our codes are publicly available on https://github.com/zjtfo/SP_Consistency_By_KD.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Luo, Y., Yang, B., Xu, D., et al.: A survey: complex knowledge base question answering. In: Proceedings of ICICSE, pp. 46–52. IEEE (2022)

    Google Scholar 

  2. Cao, S., Shi, J., Pan, L., et al.: KQA Pro: a dataset with explicit compositional programs for complex question answering over knowledge base. In: Proceedings of ACL, pp. 6101–6119. ACL (2022)

    Google Scholar 

  3. Sun, Y., Zhang, L., Cheng, G., et al.: SPARQA: skeleton-based semantic parsing for complex questions over knowledge bases. In: Proceedings of AAAI, New York, USA, pp. 8952–8959. AAAI (2020)

    Google Scholar 

  4. Liang, P., Jordan, M.I., Klein, D.: Lambda dependency-based compositional semantics. In: Proceedings of CCL, Suzhou, China, pp. 389–446. ACL (2013)

    Google Scholar 

  5. Zhong, V., Xiong, C., Socher, R.: Seq2sql: generating structured queries from natural language using reinforcement learning. arXiv preprint arXiv:1709.00103 (2017)

  6. Veyseh, A., Dernoncourt, F., Dou, D., et al.: A joint model for definition extraction with syntactic connection and semantic consistency. In: Proceedings of AAAI, New York, USA, pp. 9098–9105. AAAI (2020)

    Google Scholar 

  7. Gangwoo, K., Hyunjae, K., Jungsoo, P., Jaewoo, K.: Learn to resolve conversational dependency: a consistency training framework for conversational question answering. In: Proceedings of IJCNLP, Bangkok, Thailand, pp. 6130–6141. ACL (2021)

    Google Scholar 

  8. Wang, A., Cho, K., Lewis, M.: Asking and answering questions to evaluate the factual consistency of summaries. In: Proceedings of ACL, Seattle, Washington, United States, pp. 5008–5020. ACL (2020)

    Google Scholar 

  9. Nan, F., et al.: Improving factual consistency of abstractive summarization via question answering. In: Proceedings of IJCNLP, Bangkok, Thailand, pp. 6881–6894. ACL (2021)

    Google Scholar 

  10. Qin, L., Xie, T., Huang, S., Chen, Q., Xu, X., Che, W.: Don’t be contradicted with anything! CI-ToD: towards benchmarking consistency for task-oriented dialogue system. In: Proceedings of EMNLP, Punta Cana, Dominican Republic, pp. 2357–2367. ACL (2021)

    Google Scholar 

  11. Dharur, S., Tendulkar, P., Batra, D., Parikh, D., Selvaraju, R.: SOrT-ing VQA models: contrastive gradient learning for improved consistency. In: Proceedings of NAACL, Mexico City, pp. 3103–3111. ACL (2021)

    Google Scholar 

  12. Yang, S., Zhou, Q., Feng, D., et al.: Diversity and consistency: exploring visual question-answer pair generation. In: Proceedings of EMNLP, Punta Cana, Dominican Republic, pp. 1053–1066. ACL (2021)

    Google Scholar 

  13. Lyu, X., Li, J., Gong, Z., et al.: Encouraging lexical translation consistency for document-level neural machine translation. In: Proceedings of EMNLP, Punta Cana, Dominican Republic, pp. 3265–3277 (2021)

    Google Scholar 

  14. Gou, J., Yu, B., Maybank, S.J., et al.: Knowledge distillation: a survey. IJCV 129(6), 1789–1819 (2021)

    Article  Google Scholar 

  15. Miller, A.H., Fisch, A., Dodge, J., Karimi, A.H., Bordes, A., Weston, J.: Key-value memory networks for directly reading documents. In: Proceedings of EMNLP, Austin, Texas, pp. 1400–1409. ACL (2016)

    Google Scholar 

  16. Saxena, A., Tripathi, A., Talukdar, P.: Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In: Proceedings of ACL, Seattle, Washington, United States, pp. 4498–4507. ACL (2020)

    Google Scholar 

  17. Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93417-4_38

    Chapter  Google Scholar 

  18. Dey R., Salem F.M.: Gate-variants of gated recurrent unit (GRU) neural networks. In: Proceedings of MWSCAS, Boston, USA, pp. 1597-1600. IEEE (2017)

    Google Scholar 

  19. Dong, L., Lapata, M.: Language to logical form with neural attention. In: Proceedings of ACL, Berlin, Germany, pp. 33–43. ACL (2016)

    Google Scholar 

  20. Lewis, M., et al.: Bart: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In: Proceedings of ACL, Seattle, Washington, United States, pp. 7871–7880. ACL (2020)

    Google Scholar 

  21. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of ICLR, San Diego, USA (2015)

    Google Scholar 

  22. Xu, K., Wu, L., Wang, Z., Yu, M., Chen, L., Sheinin, V.: Exploiting rich syntactic information for semantic parsing with graphtosequence model. In: Proceedings of EMNLP, Brussels, Belgium, pp. 918–924. ACL (2018)

    Google Scholar 

  23. Agrawal, P., Dalmia, A., Jain, P., Bansal, A., Mittal, A., Sankaranarayanan, K.: Unified semantic parsing with weak supervision. In: Proceedings of ACL, Florence, Italy, pp. 4801–4810. ACL (2019)

    Google Scholar 

  24. Cao, R., et al.: Unsupervised dual paraphrasing for two-stage semantic parsing. In: Proceedings of ACL, Seattle, Washington, United States, pp. 6806–6817. ACL (2020)

    Google Scholar 

  25. Aghaei, S., Raad, E., Fensel, A.: Question answering over knowledge graphs: a case study in tourism. IEEE Access 10, 69788–69801 (2022)

    Article  Google Scholar 

  26. Oren, I., Herzig, J., Gupta, N., Gardner, M., Berant, J.: Improving compositional generalization in semantic parsing. In: Proceedings of EMNLP, Punta Cana, Dominican Republic, pp. 2482–2495. ACL (2020)

    Google Scholar 

  27. Lukovnikov, D., Daubener, S., Fischer, A.: Detecting compositionally out-of-distribution examples in semantic parsing. In: Proceedings of EMNLP, Punta Cana, Dominican Republic, pp. 591–598. ACL (2021)

    Google Scholar 

  28. Rashid, A., Lioutas, V., Rezagholizadeh, M.: Mate-kd: masked adversarial text, a companion to knowledge distillation. In: Proceedings of IJCNLP, Bangkok, Thailand, pp. 1062–1071. ACL (2021)

    Google Scholar 

  29. Wang, X., Jiang, Y., Bach, N., Wang, T., Huang, F., Tu, K.: Structure-level knowledge distillation for multilingual sequence labeling. In: Proceedings of ACL, Seattle, Washington, United States, pp. 3317–3330. ACL (2020)

    Google Scholar 

  30. Aguilar, G., Ling, Y., Zhang, Y., et al.: Knowledge distillation from internal representations. In: Proceedings of AAAI, New York, USA, pp. 7350–7357. AAAI (2020)

    Google Scholar 

  31. Mirzadeh, S.I., Farajtabar, M., Li, A., Levine, N., Matsukawa, A., Ghasemzadeh, H.: Improved knowledge distillation via teacher assistant. In: Proceedings of AAAI, New York, USA, pp. 5191–5198. AAAI (2020)

    Google Scholar 

  32. Yang, Z., Liu, J., Huang, J., et al.: Cross-modal contrastive distillation for instructional activity anticipation. In: Proceedings of ICPR, Montreal, QC, Canada, pp. 5002–5009. IEEE (2022)

    Google Scholar 

  33. Li, L., Lin, Y., Ren, S., Li, P., Zhou, J., Sun, X.: Dynamic knowledge distillation for pre-trained language models. In: Proceedings of EMNLP, Punta Cana, Dominican Republic, pp. 379–389. ACL (2021)

    Google Scholar 

  34. Ramamurthy, P., Aakur, S.N.: ISD-QA: iterative distillation of commonsense knowledge from general language models for unsupervised question answering. In: Proceedings of ICPR, Montreal, QC, Canada, pp. 1229–1235. IEEE (2022)

    Google Scholar 

  35. Luo, X., Chen, J., Song, T., et al.: Semi-supervised medical image segmentation through dual-task consistency. In: Proceedings of AAAI, pp. 8801–8809. AAAI (2021)

    Google Scholar 

  36. Panthaplackel, S., Li, J.J., Gligoric, M., et al.: Deep just-in-time inconsistency detection between comments and source code. In: Proceedings of AAAI, pp. 427–435. AAAI (2021)

    Google Scholar 

Download references

Acknowledgement

This work is supported by the National Key R &D Program of China (2020AAA0105203).

Author information

Authors and Affiliations

  1. Beijing University of Chemical Technology, Beijing, 100029, China

    Jun Zou & Jing Wan

  2. Tsinghua University, Beijing, 100084, China

    Shulin Cao & Lei Hou

  3. Beijing Caizhi Technology Co., Ltd., Beijing, 100081, China

    Jianjun Xu

Authors
  1. Jun Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shulin Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jing Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lei Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jianjun Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Wan .

Editor information

Editors and Affiliations

  1. Peng Cheng Laboratory, Shenzhen, China

    Xiangyu Song

  2. China University of Geosciences, Wuhan, China

    Ruyi Feng

  3. China University of Geosciences, Wuhan, China

    Yunliang Chen

  4. Deakin University, Burwood, VIC, Australia

    Jianxin Li

  5. University of Exeter, Exeter, UK

    Geyong Min

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zou, J., Cao, S., Wan, J., Hou, L., Xu, J. (2024). Improving the Consistency of Semantic Parsing in KBQA Through Knowledge Distillation. In: Song, X., Feng, R., Chen, Y., Li, J., Min, G. (eds) Web and Big Data. APWeb-WAIM 2023. Lecture Notes in Computer Science, vol 14333. Springer, Singapore. https://doi.org/10.1007/978-981-97-2387-4_25

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-2387-4_25

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-2386-7

  • Online ISBN: 978-981-97-2387-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation