Skip to main content
SpringerLink
Log in

Interactive knowledge-enhanced attention network for answer selection

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Answer selection which aims to select the most appropriate answers from a set of candidate answers plays a crucial role in various applications such as question answering (QA) and information retrieval. Recently, remarkable progress has been achieved on matching sequence pairs by deep neural networks. However, most of them focus on learning semantic representations for the contexts of QA pairs while the background information and facts beyond the context are neglected. In this paper, we propose an interactive knowledge-enhanced attention network for answer selection (IKAAS), which interactively learns the sentence representations of query–answer pairs by simultaneously considering the external knowledge from knowledge graphs and textual information of QA pairs. In this way, we can exploit the semantic compositionality of the input sequences and capture more comprehensive knowledge-enriched intra-document features within the question and answer. Specifically, we first propose a context-aware attentive mechanism to learn the knowledge representations guided by the corresponding context. The relations between the question and answer are then captured by computing the question–answer alignment matrix. We further employ self-attention to capture the global features of the input sequences, which are then used to calculate the relevance score of the question and answer. Experimental results on four real-life datasets demonstrate that IKAAS outperforms the compared methods. In addition, a series of analyses shows the robust superiority and the extensive applicability of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. www.110.com/ask/.

  2. http://cs.jhu.edu/~xuchen/packages/jacana-qa-naacl2013-data-results.tar.bz2.

  3. https://github.com/shuzi/insuranceQA.

  4. http://alt.qcri.org/semeval2016/task3/index.php?id=data-and-tools.

  5. http://www.110.com/ask/.

  6. https://github.com/siatnlp/LegalQA.

  7. https://github.com/fxsjy/jieba.

  8. http://nlp.stanford.edu/data/glove.42B.300d.zip.

  9. https://github.com/Embedding/Chinese-Word-Vectors.

References

  1. Jijkoun V, de Rijke M et al (2005) Recognizing textual entailment using lexical similarity. In: Proceedings of the PASCAL challenges workshop on recognising textual entailment. Citeseer, pp 73–76

  2. Wang M (2006) A survey of answer extraction techniques in factoid question answering. Comput Linguist 1(1):1–14

    Google Scholar 

  3. Wang M, Smith NA, Mitamura T (2007) What is the jeopardy model? A quasi-synchronous grammar for QA. In: Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP-CoNLL)

  4. Severyn A, Moschitti A (2015) Learning to rank short text pairs with convolutional deep neural networks. In: Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval. ACM, pp 373–382

  5. Wang D, Nyberg E (2015) A long short-term memory model for answer sentence selection in question answering. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 2: short papers), vol 2, pp 707–712

  6. Yin W, Schütze H, Xiang B, Zhou B (2016) Abcnn: attention-based convolutional neural network for modeling sentence pairs. Trans Assoc Comput Linguist 4:259–272

    Article  Google Scholar 

  7. Yang L, Ai Q, Guo J, Croft WB (2016) ANMM: ranking short answer texts with attention-based neural matching model. In: Proceedings of the 25th ACM international on conference on information and knowledge management. ACM, pp 287–296

  8. Santos CD, Tan M, Xiang B, Zhou B (2016) Attentive pooling networks. arXiv preprint arXiv:1602.03609

  9. Prager J et al (2007) Open-domain question–answering. Found Trends® Inf Retr 1(2):91–231

  10. Soubbotin MM, Soubbotin SM (2001) Patterns of potential answer expressions as clues to the right answers. In: TREC. Citeseer

  11. Heilman M, Smith NA (2010) Tree edit models for recognizing textual entailments, paraphrases, and answers to questions. In: Human language technologies: the 2010 annual conference of the North American chapter of the association for computational linguistics. Association for Computational Linguistics, pp 1011–1019

  12. Yih W, Chang M-W, Meek C, Pastusiak A (2013) Question answering using enhanced lexical semantic models. In: Proceedings of the 51st annual meeting of the association for computational linguistics (volume 1: long papers), vol 1, pp 1744–1753

  13. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  14. Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using RNN encoder–decoder for statistical machine translation. arXiv preprint arXiv:1406.1078

  15. Zhang H, Li J, Ji Y, Yue H (2016) Understanding subtitles by character-level sequence-to-sequence learning. IEEE Trans Ind Inf 13(2):616–624

    Article  Google Scholar 

  16. Kim Y (2014) Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882

  17. Tan M, Dos Santos C, Xiang B, Zhou B (2016) Improved representation learning for question answer matching. In: Proceedings of the 54th annual meeting of the association for computational linguistics (volume 1: long papers), vol 1, pp 464–473

  18. Rao J, He H, Lin J (2016) Noise-contrastive estimation for answer selection with deep neural networks. In: Proceedings of the 25th ACM international on conference on information and knowledge management. ACM, pp 1913–1916

  19. Tan M, Santos CD, Xiang B, Zhou B (2015) Lstm-based deep learning models for non-factoid answer selection. arXiv preprint arXiv:1511.04108

  20. Wang B, Liu K, Zhao J (2016) Inner attention based recurrent neural networks for answer selection. In: Proceedings of the 54th annual meeting of the association for computational linguistics (volume 1: long papers), vol 1, pp 1288–1297

  21. Zhang X, Li S, Sha L, Wang H (2017) Attentive interactive neural networks for answer selection in community question answering. In AAAI, pp 3525–3531

  22. Shen G, Yang Y, Deng Z-H (2017) Inter-weighted alignment network for sentence pair modeling. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 1179–1189

  23. Tay Y, Luu AT, Hui SC (2018) Co-stack residual affinity networks with multi-level attention refinement for matching text sequences. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 4492–4502

  24. Sha L, Zhang X, Qian F, Chang B, Sui Z (2018) A multi-view fusion neural network for answer selection. In: Thirty-second AAAI conference on artificial intelligence

  25. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008

  26. Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Advances in neural information processing systems, pp 2787–2795

  27. Toutanova K, Chen D, Pantel P, Poon H, Choudhury P, Gamon M (2015) Representing text for joint embedding of text and knowledge bases. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 1499–1509

  28. Ahn S, Choi H, Pärnamaa T, Bengio Y (2016) A neural knowledge language model. arXiv preprint arXiv:1608.00318

  29. Yang B, Mitchell T (2017) Leveraging knowledge bases in LSTMs for improving machine reading. In: Proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: long papers), vol 1, pp 1436–1446

  30. Xin J, Lin Y , Liu Z , Sun M (2018) Improving neural fine-grained entity typing with knowledge attention. In: Thirty-second AAAI conference on artificial intelligence

  31. Han X, Liu Z, Sun M (2018) Neural knowledge acquisition via mutual attention between knowledge graph and text. In: Thirty-second AAAI conference on artificial intelligence

  32. Kalchbrenner N, Grefenstette E, Blunsom P (2014) A convolutional neural network for modelling sentences. In: Proceedings of the 52nd annual meeting of the association for computational linguistics (volume 1: long papers), pp 655–665

  33. Kondrak G (2005) N-gram similarity and distance. In: International symposium on string processing and information retrieval. Springer, pp 115–126

  34. Tay Y, Phan MC, Tuan LA, Hui SC (2017) Learning to rank question answer pairs with holographic dual LSTM architecture. In: Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval. ACM, pp 695–704

  35. Yao X, Van Durme B, Callison-Burch C, Clark P (2013) Answer extraction as sequence tagging with tree edit distance. In: Proceedings of the 2013 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 858–867

  36. Feng M, Xiang B, Glass MR, Wang L, Zhou B (2015) Applying deep learning to answer selection: a study and an open task. In: 2015 IEEE workshop on automatic speech recognition and understanding (ASRU). IEEE, pp 813–820

  37. Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD international conference on management of data. ACM, pp 1247–1250

  38. Xu B, Xu Y, Liang J, Xie C, Liang B, Cui W, Xiao Y (2017) Cn-dbpedia: a never-ending Chinese knowledge extraction system. In: International conference on industrial, engineering and other applications of applied intelligent systems. Springer, pp 428–438

  39. He H, Lin J (2016) Pairwise word interaction modeling with deep neural networks for semantic similarity measurement. In: Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 937–948

  40. Barrón-Cedeno A, Da San Martino G, Joty S, Moschitti A, Al-Obaidli F, Romeo S, Tymoshenko K, Uva A (2016) Convkn at semeval-2016 task 3: answer and question selection for question answering on Arabic and English fora. In: Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), pp 896–903

  41. Mohtarami M, Belinkov Y, Hsu W-N, Zhang Y, Lei T, Bar K, Cyphers S, Glass J (2016) Sls at semeval-2016 task 3: neural-based approaches for ranking in community question answering. In: Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), pp 828–835

  42. Da San Martino G, Barrón Cedeño A, Romeo S, Uva A, Moschitti A (2016) Learning to re-rank questions in community question answering using advanced features. In: Proceedings of the 25th ACM international on conference on information and knowledge management. ACM, pp 1997–2000

  43. Wang C, Jiang F, Yang H (2017) A hybrid framework for text modeling with convolutional RNN. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 2061–2069

  44. Wang Z, Hamza W, Florian R (2017) Bilateral multi-perspective matching for natural language sentences. In: Proceedings of the 26th international joint conference on artificial intelligence. AAAI Press, pp 4144–4150

  45. Tymoshenko K, Bonadiman D, Moschitti A (2017) Ranking kernels for structures and embeddings: a hybrid preference and classification model. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 897–902

  46. Tay Y, Tuan LA, Hui SC (2018) Cross temporal recurrent networks for ranking question answer pairs. In: Thirty-second AAAI conference on artificial intelligence

  47. Deng Y, Shen Y, Yang M, Li Y, Du N, Fan W, Lei K (2018) Knowledge as a bridge: improving cross-domain answer selection with external knowledge. In: Proceedings of the 27th international conference on computational linguistics, pp 3295–3305

  48. Sachan M, Xing E (2018) Self-training for jointly learning to ask and answer questions. In: Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long papers), vol 1, pp 629–640

  49. Tran NK, Niedereée C (2018) Multihop attention networks for question answer matching. In: The 41st international ACM SIGIR conference on research and development in information retrieval. ACM, pp 325–334

  50. Kim S, Kang I, Kwak N (2019) Semantic sentence matching with densely-connected recurrent and co-attentive information. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 6586–6593

  51. Song Y, Hu QV, He L (2019) P-CNN: enhancing text matching with positional convolutional neural network. Knowl Based Syst 169:67–79

    Article  Google Scholar 

  52. Lai TM, Bui T, Li S (2018) A review on deep learning techniques applied to answer selection. In: Proceedings of the 27th international conference on computational linguistics, pp 2132–2144

Download references

Acknowledgements

This work was partially supported by the National Science Foundation of China under Grant No. 61902385, the CAS Pioneer Hundred Talents Program under Grant No. 2017-063, and the SIAT Innovation Program for Excellent Young Researchers program under Grant No. Y8G027. Min Yang was sponsored by CCF-Tencent Open Research Fund.

Author information

Authors and Affiliations

  1. Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China

    Weiyi Huang, Qiang Qu & Min Yang

  2. University of Chinese Academy of Sciences, Beijing, China

    Weiyi Huang

Authors
  1. Weiyi Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qiang Qu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Min Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiang Qu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, W., Qu, Q. & Yang, M. Interactive knowledge-enhanced attention network for answer selection. Neural Comput & Applic 32, 11343–11359 (2020). https://doi.org/10.1007/s00521-019-04630-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04630-x

Keywords

Search

Navigation