Skip to main content
SpringerLink
Log in

Bi-directional LSTM Model with Symptoms-Frequency Position Attention for Question Answering System in Medical Domain

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Online medical intelligent question answering system plays an increasingly important role as a supplement of the traditional medical service systems. The purpose is to provide quick and concise feedback on users’ questions through natural language. The technical challenges mainly lie in symptom semantic understanding and representation of users’ description. Although the performance of phrase-level and numerous attention models have been improved, the lexical gap and position information are not emphasized enough. This paper combines word2vec and the Chinese Ci-Lin [it is a dictionary that plays an auxiliary role in word2vec where processing Chinese (https://www.ltp-cloud.com/download)] to propose synonyms-subject replacement mechanism (i.e., map common words as kernel words) and realize the normalization of the semantic representation; Meanwhile, based on the bi-directional LSTM model, this paper introduces a method of the combination of adaptive weight assignment techniques and positional context, enhancing attention to the typical symptoms of the disease. More attention weight is given to the neighboring words and propose the Bi-directional Long Short Term Memory Model with Symptoms-Frequency Position Attention (BLSTM-SFPA). The good performance of the BLSTM-SFPA model has been demonstrated in comparative experiments on the medical field dataset (MED-QA and GD-QA).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

All data generated or analysed during this study are included in this published article and can be download at the link MED-QA (http://60.205.200.136:8080/QA/MED_QA.zip) and GD-QA (http://60.205.200.136:8080/QA/GD_QA.zip). There are no constraints when you utilize it in scientific research.

Notes

  1. http://www.haodf.com.

  2. http://www.xywy.com.

References

  1. Abacha AB, Zweigenbaum P (2015) MEANS: a medical question-answering system combining NLP techniques and semantic web technologies. Inf Process Manag 51(5):570–594

    Article  Google Scholar 

  2. Bahdanau D, Cho K, Bengio Y (2014) Neural Machine translation by jointly learning to align and translate. arXiv:1409.0473

  3. Berger A, Lafferty J (1999) Information retrieval as statistical translation. In: Proceedings of the 22nd annual international ACM SIGIR conference on research and development in information retrieval, ACM, pp 22–229

  4. Berger A, Caruana R, Cohn D, Freitag D, Mittal V (2000) Bridging the lexical chasm: statistical approaches to answer-finding. In: Proceedings of the 23rd annual international ACM SIGIR conference on research and development in information retrieval, ACM, pp 192–199

  5. Cai L, Zhou G, Liu K, Zhao J (2011) Learning the latent topics for question retrieval in community qa. In: Proceedings of 5th international joint conference on natural language processing, pp 273–281

  6. Cho K, Van Merrienboer B, Gulcehrc C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv:1406.1078

  7. Guo D, Li W, Fang X (2017) Capturing temporal structures for video captioning by spatio-temporal contexts and channel attention mechanism. Neural Process Lett 46(1):313–328

    Article  Google Scholar 

  8. Jeon J, Croft WB, Lee JH (2005) Finding similar questions in large question and answer archives. In: Proceedings of the 14th ACM international conference on Information and knowledge management. ACM, pp 84–90

  9. Ji Z, Xu F, Wang B, He B (2012) Question-answer topic model for question retrieval in community question answering. In: Proceedings of the 21st ACM international conference on Information and knowledge management. ACM, pp 2471–2474

  10. Jiao Y, Zhang Y, Chen X, Yin E, Jin J, Wang X, Cichocki A (2018) Spare group representation model for motor imagery EEG classification. IEEE J Biomed Health Inform 23(2):631–641

    Article  Google Scholar 

  11. Jin Z, Zhou G, Gao D, Zhang Y (2018) Eeg classification using sparse Bayesian extreme learning machine for brain–computer interface. Neural Comput Appl 1:1–9

    Google Scholar 

  12. Li H, Min MR, Ge Y, Kadav A (2017) A context-aware attention network for interactive question answering. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 927–935

  13. Liu B, An X, Huang JX (2015) Using term location information to enhance probabilistic information retrieval. In: Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval. ACM, pp 883–886

  14. Salton G, Wong A, Yang CS (1975) A vector space model for automatic indexing. Commun ACM 18(11):613–620

    Article  Google Scholar 

  15. Santos CD, Tan M, Xiang B, Zhou B (2016) Attentive pooling networks. arXiv:1602.03609

  16. Severyn A, Moschitti A (2015) Learning to rank short text pairs with convolutional deep neural networks. In: Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval. ACM, pp 373–382

  17. Singhal A, Salton G, Mitra M, Buckley C (1996) Document length normalization. Inf Process Manag 32(5):619–633

    Article  Google Scholar 

  18. Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. arXiv:1409.3215

    Google Scholar 

  19. Tan M, Santos Cd, Xiang B, Zhou B (2015) Lstm-based deep learning models for non-factoid answer selection. arXiv:1511.04108

  20. Wang B, Liu K, Zhao J (2016) Inner attention based recurrent neural networks for answer selection. In: Proceedings of the 54th annual meeting of the association for computational linguistics (volume 1: long papers), vol 1, pp 1288–1297

  21. Wang D, Nyberg E (2015) A long short-term memory model for answer sentence selection in question answering. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 2: short papers), vol 2, pp 707–712

  22. Wu CH, Liu CH, Su PH (2015) Sentence extraction with topic modelling for question-answer pair generation. Soft Comput 19(1):39–46

    Article  Google Scholar 

  23. Xue X, Jeon J, Croft WB (2008) Retrieval models for question and answer archives. In: Proceedings of the 31st annual international ACM SIGIR conference on research and development in information retrieval. ACM, pp 475–482

  24. Yan Y, Wang Y, Gao WC, Zhang BW, Yang C, Yin XC (2018) Lstm2: multi-label ranking for document classification. Neural Process Lett 47(1):117–138

    Article  Google Scholar 

  25. Yu L, Hermann KM, Blunsom P, Pulman S (2014) Deep learning for answer sentence selection. arXiv:1412.1632

  26. Zhai C, Lafferty J (2001) A study of smoothing methods for language models applied to ad hoc information retrieval. In: International ACM SIGIR conference on research and development in information retrieval, pp 334–342

  27. Zhang Y, Wang Y, Zhou G, Jin J, Wang B, Wang X, Cichocki A (2018) Multi-kernel extreme learning machine for eeg classification in brain–computer interfaces. Expert Syst Appl 96:302–310

    Article  Google Scholar 

  28. Zhao J, Huang JX, He B (2011) Crter: using cross terms to enhance probabilistic information retrieval. In: Proceedings of the 34th international ACM SIGIR conference on research and development in information Retrieval. ACM, pp 155–164

  29. Zhou G, Cai L, Zhao J, Liu K (2011) Phrase-based translation model for question retrieval in community question answer archives. In: Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies volume 1. Association for Computation Linguistics, pp 653–662

  30. Zhou G, He T, Zhao J, Hu P (2015) Learning continuous word embedding with metadata for question retrieval in community question answering. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 1: long papers), vol 1, pp 250–259

Download references

Funding

This study is supported by National Key Technology R&D Program of China (No. 2016YFD0401205), National Natural Science Foundation of China (No. 61873027), General Project of Scientific research Plan of Beijing Municipal Education Commission (NO. KM201510011008) and Science and Technology Program of Beijing Municipal Science and Technology Commission (NO. Z191100008619007).

Author information

Authors and Affiliations

  1. National Engineering Laboratory for Agri-Product Quality Traceability, Beijing Technology and Business University, Beijing, 100048, China

    Mingwen Bi, Qingchuan Zhang, Min Zuo, Zelong Xu & Qingyu Jin

Authors
  1. Mingwen Bi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qingchuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Min Zuo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zelong Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qingyu Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qingchuan Zhang.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest regarding the publication of this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bi, M., Zhang, Q., Zuo, M. et al. Bi-directional LSTM Model with Symptoms-Frequency Position Attention for Question Answering System in Medical Domain. Neural Process Lett 51, 1185–1199 (2020). https://doi.org/10.1007/s11063-019-10136-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-019-10136-3

Keywords

Search

Navigation