Abstract
Discovering the effect of the simultaneous assumption of drugs is a very important field in medical research that could improve the effectiveness of healthcare and avoid adverse drug reactions which can cause health problems to patients. Although there are several pharmacological databases containing information on drugs, this type of information is often expressed in the form of free text. Analyzing sentences in order to extract drug-drug interactions was the objective of the DDIExtraction-2013 task. Despite the fact that the challenge took place six years ago, the interest on this task is still active and several new methods based on Recurrent Neural Networks and Attention Mechanisms have been designed. In this paper, we propose a model that combines bidirectional Long Short Term Memory (LSTM) networks with the Self-Interaction Attention Mechanism. Experimental analysis shows how this model improves the classification accuracy reducing the tendency to predict the majority class resulting in false negatives, over several input configurations.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
References
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate (2014). http://arxiv.org/abs/1409.0473. cite arxiv:1409.0473Comment. Accepted at ICLR 2015 as oral presentation
Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13(1), 281–305 (2012). http://dl.acm.org/citation.cfm?id=2503308.2188395
Björne, J., Kaewphan, S., Salakoski, T.: UTurku: drug named entity recognition and drug-drug interaction extraction using SVM classification and domain knowledge. In: Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation, SemEval 2013, pp. 651–659. Association for Computational Linguistics, Atlanta, June 2013. https://www.aclweb.org/anthology/S13-2108
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1724–1734. Association for Computational Linguistics, Doha, October 2014. https://doi.org/10.3115/v1/D14-1179, https://www.aclweb.org/anthology/D14-1179
Chowdhury, M.F.M., Lavelli, A.: FBK-irst: a multi-phase kernel based approach for drug-drug interaction detection and classification that exploits linguistic information. In: Proceedings of the 7th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2013, Atlanta, Georgia, USA, 14–15 June 2013, pp. 351–355 (2013). http://aclweb.org/anthology/S/S13/S13-2057.pdf
Du, J., Han, J., Way, A., Wan, D.: Multi-level structured self-attentions for distantly supervised relation extraction. CoRR abs/1809.00699 (2018). http://arxiv.org/abs/1809.00699
Gers, F.A., Schmidhuber, J., Cummins, F.A.: Learning to forget: continual prediction with LSTM. Neural Comput. 12, 2451–2471 (2000)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016). http://www.deeplearningbook.org
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–80 (1997). https://doi.org/10.1162/neco.1997.9.8.1735
Kadlec, R., Schmid, M., Bajgar, O., Kleindienst, J.: Text understanding with the attention sum reader network. CoRR abs/1603.01547 (2016)
Kumar, S., Anand, A.: Drug-drug interaction extraction from biomedical text using long short term memory network. CoRR abs/1701.08303 (2017)
Lee, J., et al.: BioBERT: pre-trained biomedical language representation model for biomedical text mining. arXiv preprint arXiv:1901.08746 (2019)
Li, L., Guo, Y., Qian, S., Zhou, A.: An end-to-end entity and relation extraction network with multi-head attention. In: Sun, M., Liu, T., Wang, X., Liu, Z., Liu, Y. (eds.) CCL/NLP-NABD -2018. LNCS (LNAI), vol. 11221, pp. 136–146. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01716-3_12
Li, Y., Yang, T.: Word embedding for understanding natural language: a survey. In: Srinivasan, S. (ed.) Guide to Big Data Applications. SBD, vol. 26, pp. 83–104. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-53817-4_4
Liu, S., Tang, B., Chen, Q., Wang, X.: Drug-drug interaction extraction via convolutional neural networks. Comput. Math. Methods Med. 2016, 8 (2016)
Luong, M.T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025 (2015)
McDonald, R., Brokos, G., Androutsopoulos, I.: Deep relevance ranking using enhanced document-query interactions. CoRR abs/1809.01682 (2018)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Burges, C.J.C., Bottou, L., Welling, M., Ghahramani, Z., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 26, pp. 3111–3119. Curran Associates, Inc. (2013). http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf
Tarwani, K.M., Edem, S.: Survey on recurrent neural network in natural language processing. Int. J. Eng. Trends Technol. 48, 301–304 (2017). https://doi.org/10.14445/22315381/IJETT-V48P253
Nagi, J., et al.: Max-pooling convolutional neural networks for vision-based hand gesture recognition. In: 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), pp. 342–347, November 2011. https://doi.org/10.1109/ICSIPA.2011.6144164
Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of the 2014 conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)
Quan, C., Hua, L., Sun, X., Bai, W.: Multichannel convolutional neural network for biological relation extraction. BioMed. Res. Int. 2016, 10 (2016)
Raffel, C., Ellis, D.P.W.: Feed-forward networks with attention can solve some long-term memory problems. CoRR abs/1512.08756 (2015)
Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Sig. Process. 45(11), 2673–2681 (1997)
Segura-Bedmar, I., Martínez, P., Herrero-Zazo, M.: Lessons learnt from the DDIExtraction-2013 shared task. J. Biomed. Inform. 51, 152–164 (2014)
Suárez-Paniagua, V., Segura-Bedmar, I.: Evaluation of pooling operations in convolutional architectures for drug-drug interaction extraction. BMC Bioinform. 19, 209 (2018). https://doi.org/10.1186/s12859-018-2195-1
Vaswani, A., et al.: Attention is all you need. CoRR abs/1706.03762 (2017)
Weiss, G., Provost, F.: The effect of class distribution on classifier learning: an empirical study. Technical report, Department of Computer Science, Rutgers University (2001)
Wilcoxon, F.: Individual comparisons by ranking methods. Biom. Bull. 1(6), 80–83 (1945). http://www.jstor.org/stable/3001968
Yi, Z., et al.: Drug-drug interaction extraction via recurrent neural network with multiple attention layers. In: Cong, G., Peng, W.-C., Zhang, W.E., Li, C., Sun, A. (eds.) ADMA 2017. LNCS (LNAI), vol. 10604, pp. 554–566. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69179-4_39
Zeng, D., Liu, K., Lai, S., Zhou, G., Zhao, J.: Relation classification via convolutional deep neural network. In: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pp. 2335–2344. Dublin City University and Association for Computational Linguistics, Dublin, August 2014. https://www.aclweb.org/anthology/C14-1220
Zhang, Y., Zheng, W., Lin, H., Wang, J., Yang, Z., Dumontier, M.: Drug-drug interaction extraction via hierarchical RNNs on sequence and shortest dependency paths. Bioinformatics 34(5), 828–835 (2018)
Zhang, Y., Qi, P., Manning, C.D.: Graph convolution over pruned dependency trees improves relation extraction. CoRR abs/1809.10185 (2018)
Zheng, J., Cai, F., Shao, T., Chen, H.: Self-interaction attention mechanism-based text representation for document classification. Appl. Sci. 8(4), 613 (2018). https://doi.org/10.3390/app8040613. http://www.mdpi.com/2076-3417/8/4/613
Zheng, W., et al.: An attention-based effective neural model for drug-drug interactions extraction. BMC Bioinform. 18, 445 (2017). https://doi.org/10.1186/s12859-017-1855-x
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Putelli, L., Gerevini, A.E., Lavelli, A., Serina, I. (2019). Applying Self-interaction Attention for Extracting Drug-Drug Interactions. In: Alviano, M., Greco, G., Scarcello, F. (eds) AI*IA 2019 – Advances in Artificial Intelligence. AI*IA 2019. Lecture Notes in Computer Science(), vol 11946. Springer, Cham. https://doi.org/10.1007/978-3-030-35166-3_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-35166-3_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-35165-6
Online ISBN: 978-3-030-35166-3
eBook Packages: Computer ScienceComputer Science (R0)