Skip to main content
SpringerLink
Log in

A syntactic path-based hybrid neural network for negation scope detection

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

The automatic detection of negation is a crucial task in a wide-range of natural language processing (NLP) applications, including medical data mining, relation extraction, question answering, and sentiment analysis. In this paper, we present a syntactic path-based hybrid neural network architecture, a novel approach to identify the scope of negation in a sentence. Our hybrid architecture has the particularity to capture salient information to determine whether a token is in the scope or not, without relying on any human intervention. This approach combines a bidirectional long short-term memory (Bi-LSTM) network and a convolutional neural network (CNN). The CNN model captures relevant syntactic features between the token and the cue within the shortest syntactic path in both constituency and dependency parse trees. The Bi-LSTM learns the context representation along the sentence in both forward and backward directions. We evaluate our model on the Bioscope corpus, and get 90.82% F-score (78.31% PCS) on the abstract sub-corpus, outperforming features-dependent approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Morante R, Liekens A, Daelemans, W. Learning the scope of negation in biomedical texts. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics. 2008, 715–724

    Google Scholar 

  2. Chapman W, Bridewell W, Hanbury P, Cooper G F, Buchanan B G. A simple algorithm for identifying negated findings and diseases in discharge summaries. Journal of Biomedical Informatics, 2001, 34(5): 301–310

    Article  Google Scholar 

  3. Mutalik P G, Deshpande A, Nadkarni P M. Use of general-purpose negation detection to augment concept indexing of medical documents. Journal of the American Medical Informatics Association, 2001, 8(6): 598–609

    Article  Google Scholar 

  4. Huang Y, Lowe H J. A novel hybrid approach to automated negation detection in clinical radiology reports. Journal of the American Medical Informatics Association, 2007, 14(3): 304–311

    Article  Google Scholar 

  5. Vincze V, Szarvas G, Farkas R, Mra G, Csirik J. The BioScope corpus: biomedical texts annotated for uncertainty, negation and their scopes. BMC Bioinformatics, 2008, 9(11): S9

    Article  Google Scholar 

  6. Morante R, Daelemans W. A metalearning approach to processing the scope of negation. In: Proceedings of the Thirteenth Conference on Computational Natural Language Learning, Association for Computational Linguistics. 2009, 21–29

    Google Scholar 

  7. Zou B, Zhou G, Zhu Q. Tree kernel-based negation and speculation scope detection with structured syntactic parse features. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2013, 968–976

    Google Scholar 

  8. Abu-Jbara A, Dragomir R. Umichigan: a conditional random field model for resolving the scope of negation. In: Proceedings of the First Joint Conference on Lexical and Computational Semantics-Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation, Association for Computational Linguistics. 2012, 328–334

    Google Scholar 

  9. Agarwal S, Yu H. Biomedical negation scope detection with conditional random fields. Journal of the American Medical Informatics Association, 2010, 17(6): 696–701

    Article  Google Scholar 

  10. Lazib L, Zhao Y, Qin B, Liu T. Negation scope detection with conditional random field model. High Technology Letters, 2017, 23(2): 191–197

    Google Scholar 

  11. Cho K, Van Merrinboer B, Bahdanau D, Bengio Y. On the properties of neural machine translation: Encoder-decoder approaches. 2014, arXiv preprint arXiv: 1409.1259

    Google Scholar 

  12. Zeng D, Liu K, Chen Y, Zhao J. Distant supervision for relation extraction via piecewise convolutional neural networks. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2015, 1753–1762

    Chapter  Google Scholar 

  13. Tang D, Qin B, Liu T. Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2015, 1426–1432

    Google Scholar 

  14. Lazib L, Zhao Y, Qin B, Liu T. Negation scope detection with recurrent neural networks models in review texts. In: Proceedings of International Conference of Young Computer Scientists, Engineers and Educators. Springer, Singapore. 2016, 494–508

    Google Scholar 

  15. Lazib L, Zhao Y, Qin B, Liu T. Negation scope detection with recurrent neural networks models in review texts. International Journal of High Performance Computing and Networking. DOI 10.1504/IJHPCN. 2016.10011341

  16. Qian Z, Li P, Zhu Q, Zhou G, Luo Z, Luo W. Speculation and negation scope detection via convolutional neural networks. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2016, 815–825

    Chapter  Google Scholar 

  17. Fancellu F, Lopez A, Webber B L. Neural networks for negation scope detection. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 495–504

    Google Scholar 

  18. Schuster M, Paliwal K K. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 1997, 45(11): 2673–2681

    Article  Google Scholar 

  19. Mikolov T, Karafit M, Burget L, Černockỳ J, Khudanpur S. Recurrent neural network based language model. In: Proceedings of Eleventh Annual Conference of the International Speech Communication Association. 2010, 1045–1048

    Google Scholar 

  20. LeCun Y, Bengio Y. Convolutional networks for images, speech, and time series. The Handbook of Brain Theory and Neural Networks, 1995, 3361(10): 1995

    Google Scholar 

  21. Liu Y, Wei F, Li S, Ji H, Zhou M, Wang H. A dependency-based neural network for relation classification. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015, 285–290

    Google Scholar 

  22. Cai R, Zhang X, Wang H. Bidirectional recurrent convolutional neural network for relation classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 756–765

    Google Scholar 

  23. Xu Y, Mou L, Li G, Chen Y, Peng H, Jin Z. Classifying relations via long short-term memory networks along shortest dependency paths. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2015, 1785–1794

    Chapter  Google Scholar 

  24. Vrelid L, Velldal E, Oepen S. Syntactic scope resolution in uncertainty analysis. In: Proceedings of the 23rd International Conference on Computational Linguistics. 2010, 1379–1387

    Google Scholar 

  25. Lafferty J, McCallum A, Pereira F C. Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of Probabilistic Models for Segmenting and Labeling Sequence Data. 2001, 282–289

    Google Scholar 

  26. White J P. UWashington: negation resolution using machine learning methods. In: Proceedings of the First Joint Conference on Lexical and Computational Semantics-Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the 6th International Workshop on Semantic Evaluation. Association for Computational Linguistics. 2012, 335–339

    Google Scholar 

  27. Huang Z, Xu W, Yu K. Bidirectional LSTM-CRF models for sequence tagging. 2015, arXiv, preprint arXiv:1508.01991

    Google Scholar 

  28. Wang P, Qian Y, Soong F K, He L, Zhao H. A unified tagging solution: bidirectional LSTM recurrent neural network with word embedding. 2015, arXiv, preprint arXiv: 1511.00215

    Google Scholar 

  29. Taboada M, Anthony C, Voll K. Methods for creating semantic orientation dictionaries. In: Proceedings of the 5th Conference on Language Resources and Evaluation. 2006, 427–432

    Google Scholar 

  30. Zeng D, Liu K, Lai S, Zhou G, Zhao J. Relation classification via convolutional deep neural network. In: Proceedings of the 25th International Conference on Computational Linguistics: Technical Papers. 2014, 2335–2344

    Google Scholar 

  31. Zhang B, Su J, Xiong D, Lu Y, Duan H, Yao J. Shallow convolutional neural network for implicit discourse relation recognition. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2015, 2230–2235

    Chapter  Google Scholar 

  32. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation, 1997, 9(8): 1735–1780

    Article  Google Scholar 

  33. Graves A, Schmidhuber J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks, 2005, 18(5): 602–610

    Article  Google Scholar 

  34. Sundermeyer M, Schlter R, Ney H. LSTM neural networks for language modeling. In: Proceedings of the 13th Annual Conference of the International Speech Communication Association. 2013, 194–197

    Google Scholar 

  35. Kadari R, Zhang Y, Zhang W, Liu T. CCG supertagging with bidirectional long short-term memory networks. Natural Language Engineering, 2018, 24(1): 77–90

    Article  Google Scholar 

  36. Kadari R, Zhang Y, Zhang W, Liu T. CCG supertagging via Bidirectional LSTM-CRF neural architecture. Neurocomputing, 2018, 283: 31–37

    Article  Google Scholar 

  37. Graves A, Jaitly N. Towards end-to-end speech recognition with recurrent neural networks. In: Proceedings of the 31st International Conference on Machine Learning. 2014, 1764–1772

    Google Scholar 

  38. Sak H, Senior A W, Beaufays F. Long short-term memory recurrent neural network architectures for large scale acoustic modeling. In: Proceedings of the 15th Annual Conference of the International Speech Communication Association. 2014, 338–342

    Google Scholar 

  39. Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P. Natural language processing (almost) from scratch. Journal ofMachine Learning Research, 2011, 12(Aug): 2493–2537

    MATH  Google Scholar 

  40. Collier N, Park H S, Ogata N, Tateishi Y, Nobata C, Ohta T, Sekimizu T, Imai H, Ibushi K, Tsujii J I. The GENIA project: corpus-based knowledge acquisition and information extraction from genome research papers. In: Proceedings of the 9th Conference on European Chapter of the Association for Computational Linguistics. 1999, 271–272

    Google Scholar 

  41. Chollet F. Keras on GitHub, 2015

    Google Scholar 

  42. Klein D, Manning C D. Accurate unlexicalized parsing. In: Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics. 2003

    Google Scholar 

Download references

Acknowledgements

Project supported by the National Natural Science Foundation of China (Grant Nos. 61632011, 61772153, 71490722), Heilongjiang philosophy and social science research project (16TQD03)

Author information

Authors and Affiliations

  1. Research Center for Social Computing and Information Retrieval, Harbin Institute of Technology, Harbin, 150001, China

    Lydia Lazib, Bing Qin, Yanyan Zhao, Weinan Zhang & Ting Liu

Authors
  1. Lydia Lazib
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bing Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanyan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weinan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ting Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanyan Zhao.

Additional information

Lydia Lazib received her BS and MS degrees in Computer Science Department of Mouloud MAMMERI University of Tizi-Ouzou, Algeria in 2011 and 2013 respectively. Currently, she is a PhD student at Harbin Institute of Technology, China. Her research interests include sentiment analysis and negation detection.

Bing Qin is a full professor of the School of Computer Science, Harbin Institute of Technology, China. She is the deputy Director of Research Center for Social Computing and Information Retrieval (HITSCIR), and presided over the National 863 Project. Here research interest are information retrieval and natural language processing.

Yanyan Zhao is an associate professor in the Department of Media Technology and Art at Harbin Institute of Technology, China. Her interests include sentiment analysis and text mining. She’s a member of the Association for Computation Linguistics and the China Computer Federation.

Weinan Zhang is a Lecturer in Research Center for Social Computing and Information Retrieval, School of Computer Science and Technology, Harbin Institute of Technology, China. His research interest includes human-computer dialogue, natural language processing and information retrieval.

Ting Liu is a full professor of the School of Computer Science and technology at Harbin Institute of Technology, China. He is the director of the Research Center for Social Computing and Information Retrieval (SCIR). His research interest include Information Retrieval (IR) and Natural Language Processing (NLP).

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lazib, L., Qin, B., Zhao, Y. et al. A syntactic path-based hybrid neural network for negation scope detection. Front. Comput. Sci. 14, 84–94 (2020). https://doi.org/10.1007/s11704-018-7368-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-018-7368-6

Keywords

Search

Navigation