Editorial Notes
A corrigendum was issued for this article on October 29, 2019. You can download the corrigendum from the supplemental material section of this citation page.
ABSTRACT
Online healthcare services can offer public ubiquitous access to the medical knowledge, especially with the emergence of medical question answering websites, where patients can get in touch with doctors without going to hospital. Explainability and accuracy are two main concerns for medical question answering. However, existing methods mainly focus on accuracy and cannot provide a good explanation for retrieved medical answers. This paper proposes a novelMulti-Modal Knowledge-aware Hierarchical Attention Network (MKHAN) to effectively exploit multi-modal knowledge graph (MKG) for explainable medical question answering. MKHAN can generate path representation by composing the structural, linguistics, and visual information of entities, and infer the underlying rationale of question-answer interactions by leveraging the sequential dependencies within a path from MKG. Furthermore, a novel hierarchical attention network is proposed to discriminate the salience of paths endowing our model with explainability. We build a large-scale multi-modal medical knowledge graph andtwo real-world medical question answering datasets, the experimental results demonstrate the superior performance on our approachcompared with the state-of-the-art methods.
Supplemental Material
Available for Download
Corrigendum to "Multi-modal Knowledge-aware Hierarchical Attention Network for Explainable Medical Question Answering," by Zhang et al., Proceedings of the 27th ACM International Conference on Multimedia (MM '19).
- Asma Ben Abacha and Pierre Zweigenbaum. 2015. MEANS: A medical question-answering system combining NLP techniques and semantic Web technologies. Inf. Process. Manage. , Vol. 51 (2015), 570--594.Google ScholarDigital Library
- Antoine Bordes, Nicolas Usunier, Alberto Garcia-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data. In Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2 (NIPS'13). Curran Associates Inc., USA, 2787--2795.Google ScholarDigital Library
- Edward Choi, Mohammad Taha Bahadori, Elizabeth Searles, Catherine Coffey, Michael Thompson, James Bost, Javier Tejedor-Sojo, and Jimeng Sun. 2016. Multi-layer Representation Learning for Medical Concepts. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '16). ACM, New York, NY, USA, 1495--1504.Google ScholarDigital Library
- Edward Choi, Mohammad Taha Bahadori, Le Song, Walter F. Stewart, and Jimeng Sun. 2017. GRAM: Graph-based Attention Model for Healthcare Representation Learning. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '17). ACM, New York, NY, USA, 787--795.Google ScholarDigital Library
- Wanyun Cui, Yanghua Xiao, Haixun Wang, Yangqiu Song, Seung won Hwang, and Wei Yang Wang. 2017. KBQA: Learning Question Answering over QA Corpora and Knowledge Bases. Proceedings of the VLDB Endowment , Vol. 10 (2017), 565--576.Google ScholarDigital Library
- Jun da Hu, Shengsheng Qian, Quan Fang, and Changsheng Xu. 2018. Attentive Interactive Convolutional Matching for Community Question Answering in Social Multimedia. In ACM International Conference on Multimedia .Google Scholar
- Hanyin Fang, Fei Wu, Zhou Zhao, Xinyu Duan, Yueting Zhuang, and Martin Ester. 2016. Community-based Question Answering via Heterogeneous Social Network Learning. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI'16). AAAI Press, 122--128.Google Scholar
- Travis R. Goodwin and Sanda M. Harabagiu. 2016. Medical Question Answering for Clinical Decision Support. (2016), 297--306.Google Scholar
- Travis R. Goodwin and Sanda M. Harabagiu. 2017. Knowledge Representations and Inference Techniques for Medical Question Answering. ACM Transactions on Intelligent Systems and Technology , Vol. 9 (2017), 14:1--14:26.Google ScholarDigital Library
- Donglin Guo, Min Li, Ying Yu, Yaohang Li, Guihua Duan, Fang-Xiang Wu, and Jianxin Wang. 2018. Disease Inference with Symptom Extraction and Bidirectional Recurrent Neural Network. 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (2018), 864--868.Google Scholar
- Kalervo J"arvelin and Jaana Kek"al"ainen. 2000. IR Evaluation Methods for Retrieving Highly Relevant Documents. (2000), 41--48.Google Scholar
- Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Knowledge Graph Embedding via Dynamic Mapping Matrix. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)" . Association for Computational Linguistics, Beijing, China, 687--696.Google Scholar
- Quoc Le and Tomas Mikolov. 2014. Distributed Representations of Sentences and Documents. In Proceedings of the 31st International Conference on Machine Learning (Proceedings of Machine Learning Research), , Eric P. Xing and Tony Jebara (Eds.), Vol. 32. PMLR, Bejing, China, 1188--1196.Google Scholar
- Yaliang Li, Nan Du, Chaochun Liu, Yusheng Xie, Wei Fan, Qi Li, Jing Gao, and Huan Sun. 2017. Reliable Medical Diagnosis from Crowdsourcing: Discover Trustworthy Answers from Non-Experts. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining (WSDM '17). ACM, New York, NY, USA, 253--261.Google ScholarDigital Library
- Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning Entity and Relation Embeddings for Knowledge Graph Completion. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI'15). AAAI Press, 2181--2187.Google ScholarDigital Library
- Siqi Liu, Sidong Liu, Tom Weidong Cai, Hangyu Che, Sonia Pujol, Ron Kikinis, David Dagan Feng, and Michael J. Fulham. 2015. Multimodal Neuroimaging Feature Learning for Multiclass Diagnosis of Alzheimer's Disease. IEEE Transactions on Biomedical Engineering , Vol. 62 (2015), 1132--1140.Google ScholarCross Ref
- Fenglong Ma, Yaqing Wang, Houping Xiao, Ye Yuan, Radha Chitta, Jing Zhou, and Jing Gao. 2018a. A General Framework for Diagnosis Prediction via Incorporating Medical Code Descriptions. 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (2018), 1070--1075.Google ScholarCross Ref
- Fenglong Ma, Quanzeng You, Houping Xiao, Radha Chitta, Jing Zhou, and Jing Gao. 2018b. KAME: Knowledge-based Attention Model for Diagnosis Prediction in Healthcare. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management (CIKM '18). ACM, New York, NY, USA, 743--752.Google ScholarDigital Library
- Riccardo Miotto, Li Li, Brian A. Kidd, and Joel Dudley. 2016. Deep Patient: An Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records. In Scientific reports .Google Scholar
- Hatem Mousselly Sergieh, Teresa Botschen, Iryna Gurevych, and Stefan Roth. 2018. A Multimodal Translation-Based Approach for Knowledge Graph Representation Learning. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics. Association for Computational Linguistics, New Orleans, Louisiana, 225--234.Google Scholar
- Phuoc Nguyen, Truyen Tran, Nilmini Wickramasinghe, and Svetha Venkatesh. 2017. Deepr: A Convolutional Net for Medical Records. IEEE Journal of Biomedical and Health Informatics , Vol. 21 (2017), 22--30.Google ScholarCross Ref
- Fudong Nian, Bing-Kun Bao, Teng Li, and Changsheng Xu. 2017. Multi-Modal Knowledge Representation Learning via Webly-Supervised Relationships Mining. In Proceedings of the 25th ACM International Conference on Multimedia (MM '17). ACM, New York, NY, USA, 411--419.Google ScholarDigital Library
- L. Nie, M. Wang , L. Zhang, S. Yan, B. Zhang, and T. Chua. 2015. Disease Inference from Health-Related Questions via Sparse Deep Learning. IEEE Transactions on Knowledge and Data Engineering , Vol. 27, 8 (Aug 2015), 2107--2119.Google ScholarDigital Library
- Liang Pang, Yanyan Lan, Jiafeng Guo, Jun Xu, Shengxian Wan, and Xueqi Cheng. 2016. Text Matching As Image Recognition. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI'16). AAAI Press, 2793--2799.Google Scholar
- Trang Pham, Truyen Tran, Dinh Phung, and Svetha Venkatesh. 2016. DeepCare: A Deep Dynamic Memory Model for'zPredictive Medicine. In Proceedings, Part II, of the 20th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining - Volume 9652 (PAKDD 2016). Springer-Verlag, Berlin, Heidelberg, 30--41.Google ScholarDigital Library
- S. Qian, T. Zhang , C. Xu, and J. Shao. 2016. Multi-Modal Event Topic Model for Social Event Analysis. IEEE Transactions on Multimedia , Vol. 18, 2 (Feb 2016), 233--246. https://doi.org/10.1109/TMM.2015.2510329Google ScholarDigital Library
- Ying Shen, Yang Deng, Min Yang, Yaliang Li, Nan Du, Wei Fan, and Kai Lei. 2018. Knowledge-aware Attentive Neural Network for Ranking Question Answer Pairs. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval (SIGIR '18). ACM, New York, NY, USA, 901--904.Google Scholar
- Yikang Shen, Wenge Rong, Zhiwei Sun, Yuanxin Ouyang, and Zhang Xiong. 2015. Question/Answer Matching for CQA System via Combining Lexical and Sequential Information. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI'15). AAAI Press, 275--281.Google ScholarDigital Library
- Benjamin Shickel, Patrick James Tighe, Azra Bihorac, and Parisa Rashidi. 2018. Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis. IEEE Journal of Biomedical and Health Informatics , Vol. 22 (2018), 1589--1604.Google ScholarCross Ref
- Heung-Il Suk, Seong-Whan Lee, and Dinggang Shen. 2014. Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis. NeuroImage , Vol. 101 (2014), 569--582.Google ScholarCross Ref
- Zhu Sun, Jie Yang, Jie Zhang, Alessandro Bozzon, Long-Kai Huang, and Chi Xu. 2018. Recurrent Knowledge Graph Embedding for Effective Recommendation. In Proceedings of the 12th ACM Conference on Recommender Systems (RecSys '18). ACM, New York, NY, USA, 297--305.Google ScholarDigital Library
- Yi Tay, Minh C. Phan, Luu Anh Tuan, and Siu Cheung Hui. 2017. Learning to Rank Question Answer Pairs with Holographic Dual LSTM Architecture. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '17). ACM, New York, NY, USA, 695--704.Google ScholarDigital Library
- Yi Tay, Luu Anh Tuan, and Siu Cheung Hui. 2018. Cross Temporal Recurrent Networks for Ranking Question Answer Pairs. In Proceedings of the 32rd AAAI Conference on Artificial Intelligence .Google Scholar
- Truyen Tran, Tu Dinh Nguyen, Dinh Phung, and Svetha Venkatesh. 2015. Learning Vector Representation of Medical Objects via EMR-driven Nonnegative Restricted Boltzmann Machines (eNRBM). J. of Biomedical Informatics , Vol. 54, C (April 2015), 96--105.Google ScholarDigital Library
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. In NIPS .Google Scholar
- Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge Graph Embedding by Translating on Hyperplanes. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI'14). AAAI Press, 1112--1119.Google ScholarDigital Library
- Yu Ping Wu, Wei Chung Wu, Can Xu, and Zhoujun Li. 2018. Proceedings of the AAAI Conference on Artificial Intelligence. In AAAI .Google Scholar
- Ruobing Xie, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2017. Image-embodied Knowledge Representation Learning. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17. 3140--3146.Google ScholarCross Ref
- Chenyan Xiong, Zhuyun Dai, Jamie Callan, Zhiyuan Liu, and Russell Power. 2017. End-to-End Neural Ad-hoc Ranking with Kernel Pooling. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '17). ACM, New York, NY, USA, 55--64.Google ScholarDigital Library
- Xiaodong Zhang, Sujian Li, Lei Sha, and Houfeng Wang. 2017. Attentive Interactive Neural Networks for Answer Selection in Community Question Answering. In Proceedings of the 31th AAAI Conference on Artificial Intelligence .Google Scholar
Index Terms
Multi-modal Knowledge-aware Hierarchical Attention Network for Explainable Medical Question Answering
Recommendations
Knowledge Representations and Inference Techniques for Medical Question Answering
Regular PapersAnswering medical questions related to complex medical cases, as required in modern Clinical Decision Support (CDS) systems, imposes (1) access to vast medical knowledge and (2) sophisticated inference techniques. In this article, we examine the ...
Medical knowledge-based network for Patient-oriented Visual Question Answering
AbstractVisual Question Answering (VQA) systems have achieved great success in general scenarios. In medical domain, VQA systems are still in their infancy as the datasets are limited by scale and application scenarios. Current medical VQA ...
Highlights- We introduce a new Patient-oriented medical VQA dataset (P-VQA).
- P-VQA covers ...
A medical question answering system using large language models and knowledge graphs
AbstractQuestion answering systems have become prominent in all areas, while in the medical domain it has been challenging because of the abundant domain knowledge. Retrieval based approach has become promising as large pretrained language models come ...
Comments