Skip to main content

End-to-End Fine-Grained Neural Entity Recognition of Patients, Interventions, Outcomes

  • Conference paper
  • First Online:
Experimental IR Meets Multilinguality, Multimodality, and Interaction (CLEF 2021)

Abstract

PICO recognition is an information extraction task for detecting parts of text describing Participant (P), Intervention (I), Comparator (C), and Outcome (O) (PICO elements) in clinical trial literature. Each PICO description is further decomposed into finer semantic units. For example, in the sentence ‘The study involved 242 adult men with back pain.’, the phrase ‘242 adult men with back pain’ describes the participant, but this coarse-grained description is further divided into finer semantic units. The term ‘242’ shows “sample size” of the participants, ‘adult’ shows “age”, ‘men’ shows “sex”, and ‘back pain’ show the participant “condition”. Recognizing these fine-grained PICO entities in health literature is a challenging named-entity recognition (NER) task but it can help to fully automate systematic reviews (SR). Previous approaches concentrated on coarse-grained PICO recognition but focus on the fine-grained recognition still lacks. We revisit the previously unfruitful neural approaches to improve recognition performance for the fine-grained entities. In this paper, we test the feasibility and quality of multitask learning (MTL) to improve fine-grained PICO recognition using a related auxiliary task and compare it with single-task learning (STL). As a consequence, our end-to-end neural approach improves the state-of-the-art (SOTA) F1 score from 0.45 to 0.54 for the “participant” entity and from 0.48 to 0.57 for the “outcome” entity without any handcrafted features. We inspect the models to identify where they fail and how some of these failures are linked to the current benchmark data.

Supported by HES-SO Valais-Wallis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://ebm-nlp.herokuapp.com/.

  2. 2.

    https://paperswithcode.com/sota/participant-intervention-comparison-outcome.

  3. 3.

    https://ebm-nlp.herokuapp.com/#Leaderboard.

  4. 4.

    A single document consists of a title and an abstract.

  5. 5.

    https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6174533/bin/NIHMS988059-supplement-Appendix.pdf.

  6. 6.

    https://github.com/anjani-dhrangadhariya/multitask-pico-detection.

References

  1. Beltagy, I., Lo, K., Cohan, A.: Scibert: a pretrained language model for scientific text. arXiv preprint arXiv:1903.10676 (2019)

  2. Boudin, F., Nie, J.Y., Bartlett, J.C., Grad, R., Pluye, P., Dawes, M.: Combining classifiers for robust PICO element detection. BMC Med. Inform. Decis. Mak. 10(1), 1–6 (2010)

    Article  Google Scholar 

  3. Boudin, F., Nie, J.Y., Dawes, M.: Clinical information retrieval using document and PICO structure. In: Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 822–830 (2010)

    Google Scholar 

  4. Boudin, F., Shi, L., Nie, J.-Y.: Improving medical information retrieval with PICO element detection. In: Gurrin, C., et al. (eds.) ECIR 2010. LNCS, vol. 5993, pp. 50–61. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12275-0_8

  5. Caruana, R.: Multitask learning. Mach. Learn. 28(1), 41–75 (1997)

    Article  MathSciNet  Google Scholar 

  6. Chabou, S., Iglewski, M.: Combination of conditional random field with a rule based method in the extraction of PICO elements. BMC Med. Inform. Decis. Mak. 18(1), 128 (2018)

    Article  Google Scholar 

  7. Chung, G.Y.C.: Towards identifying intervention arms in randomized controlled trials: extracting coordinating constructions. J. Biomed. Inform. 42(5), 790–800 (2009)

    Article  Google Scholar 

  8. Dawes, M., Pluye, P., Shea, L., Grad, R., Greenberg, A., Nie, J.Y.: The identification of clinically important elements within medical journal abstracts: patient\(\_\)population\(\_\)problem, exposure\(\_\)intervention, comparison, outcome, duration and results (PECODR). J. Innovation Health Inf. 15(1), 9–16 (2007)

    Article  Google Scholar 

  9. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

  10. Dror, R., Baumer, G., Shlomov, S., Reichart, R.: The hitchhiker’s guide to testing statistical significance in natural language processing. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1383–1392 (2018)

    Google Scholar 

  11. Fei, H., Ren, Y., Ji, D.: Dispatched attention with multi-task learning for nested mention recognition. Inf. Sci. 513, 241–251 (2020)

    Article  Google Scholar 

  12. Fuhr, N.: Some common mistakes in IR evaluation, and how they can be avoided. In: ACM SIGIR Forum, vol. 51, pp. 32–41. ACM New York, NY, USA (2018)

    Google Scholar 

  13. He, Z., Tao, C., Bian, J., Dumontier, M., Hogan, W.R.: Semantics-powered healthcare engineering and data analytics (2017)

    Google Scholar 

  14. Hilfiker, R., et al.: Exercise and other non-pharmaceutical interventions for cancer-related fatigue in patients during or after cancer treatment: a systematic review incorporating an indirect-comparisons meta-analysis. Br. J. Sports Med. 52(10), 651–658 (2018)

    Article  Google Scholar 

  15. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  16. Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991 (2015)

  17. Jaseena, K., David, J.M.: Issues, challenges, and solutions: big data mining. CS IT-CSCP 4(13), 131–140 (2014)

    Google Scholar 

  18. Jin, D., Szolovits, P.: PICO element detection in medical text via long short-term memory neural networks. In: Proceedings of the BioNLP 2018 workshop, pp. 67–75 (2018)

    Google Scholar 

  19. Joshi, A., Karimi, S., Sparks, R., Paris, C., MacIntyre, C.R.: A comparison of word-based and context-based representations for classification problems in health informatics. arXiv preprint arXiv:1906.05468 (2019)

  20. Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., Moher, D.: Evidence summaries: the evolution of a rapid review approach. Syst. Rev. 1(1), 1–9 (2012)

    Article  Google Scholar 

  21. Nye, B., et al.: A corpus with multi-level annotations of patients, interventions and outcomes to support language processing for medical literature. In: Proceedings of the conference. Association for Computational Linguistics. Meeting. vol. 2018, p. 197. NIH Public Access (2018)

    Google Scholar 

  22. Ruder, S.: An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098 (2017)

  23. Russell, R., et al.: Systematic review methods. In: Issues and Challenges in Conducting Systematic Reviews to Support Development of Nutrient Reference Values: Workshop Summary Nutrition Research Series, vol. 2 (2009)

    Google Scholar 

  24. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)

    Google Scholar 

  25. Xu, R., Garten, Y., Supekar, K.S., Das, A.K., Altman, R.B., Garber, A.M., et al.: Extracting subject demographic information from abstracts of randomized clinical trial reports. In: Medinfo 2007: Proceedings of the 12th World Congress on Health (Medical) Informatics; Building Sustainable Health Systems, p. 550. IOS Press (2007)

    Google Scholar 

  26. Zhang, T., Yu, Y., Mei, J., Tang, Z., Zhang, X., Li, S.: Unlocking the power of deep PICO extraction: Step-wise medical NER identification. arXiv preprint arXiv:2005.06601 (2020)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Anjani Dhrangadhariya .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Dhrangadhariya, A., Aguilar, G., Solorio, T., Hilfiker, R., Müller, H. (2021). End-to-End Fine-Grained Neural Entity Recognition of Patients, Interventions, Outcomes. In: Candan, K.S., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2021. Lecture Notes in Computer Science(), vol 12880. Springer, Cham. https://doi.org/10.1007/978-3-030-85251-1_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-85251-1_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-85250-4

  • Online ISBN: 978-3-030-85251-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics