Abstract
PICO recognition is an information extraction task for detecting parts of text describing Participant (P), Intervention (I), Comparator (C), and Outcome (O) (PICO elements) in clinical trial literature. Each PICO description is further decomposed into finer semantic units. For example, in the sentence ‘The study involved 242 adult men with back pain.’, the phrase ‘242 adult men with back pain’ describes the participant, but this coarse-grained description is further divided into finer semantic units. The term ‘242’ shows “sample size” of the participants, ‘adult’ shows “age”, ‘men’ shows “sex”, and ‘back pain’ show the participant “condition”. Recognizing these fine-grained PICO entities in health literature is a challenging named-entity recognition (NER) task but it can help to fully automate systematic reviews (SR). Previous approaches concentrated on coarse-grained PICO recognition but focus on the fine-grained recognition still lacks. We revisit the previously unfruitful neural approaches to improve recognition performance for the fine-grained entities. In this paper, we test the feasibility and quality of multitask learning (MTL) to improve fine-grained PICO recognition using a related auxiliary task and compare it with single-task learning (STL). As a consequence, our end-to-end neural approach improves the state-of-the-art (SOTA) F1 score from 0.45 to 0.54 for the “participant” entity and from 0.48 to 0.57 for the “outcome” entity without any handcrafted features. We inspect the models to identify where they fail and how some of these failures are linked to the current benchmark data.
Supported by HES-SO Valais-Wallis.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
- 4.
A single document consists of a title and an abstract.
- 5.
- 6.
References
Beltagy, I., Lo, K., Cohan, A.: Scibert: a pretrained language model for scientific text. arXiv preprint arXiv:1903.10676 (2019)
Boudin, F., Nie, J.Y., Bartlett, J.C., Grad, R., Pluye, P., Dawes, M.: Combining classifiers for robust PICO element detection. BMC Med. Inform. Decis. Mak. 10(1), 1–6 (2010)
Boudin, F., Nie, J.Y., Dawes, M.: Clinical information retrieval using document and PICO structure. In: Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 822–830 (2010)
Boudin, F., Shi, L., Nie, J.-Y.: Improving medical information retrieval with PICO element detection. In: Gurrin, C., et al. (eds.) ECIR 2010. LNCS, vol. 5993, pp. 50–61. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12275-0_8
Caruana, R.: Multitask learning. Mach. Learn. 28(1), 41–75 (1997)
Chabou, S., Iglewski, M.: Combination of conditional random field with a rule based method in the extraction of PICO elements. BMC Med. Inform. Decis. Mak. 18(1), 128 (2018)
Chung, G.Y.C.: Towards identifying intervention arms in randomized controlled trials: extracting coordinating constructions. J. Biomed. Inform. 42(5), 790–800 (2009)
Dawes, M., Pluye, P., Shea, L., Grad, R., Greenberg, A., Nie, J.Y.: The identification of clinically important elements within medical journal abstracts: patient\(\_\)population\(\_\)problem, exposure\(\_\)intervention, comparison, outcome, duration and results (PECODR). J. Innovation Health Inf. 15(1), 9–16 (2007)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Dror, R., Baumer, G., Shlomov, S., Reichart, R.: The hitchhiker’s guide to testing statistical significance in natural language processing. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1383–1392 (2018)
Fei, H., Ren, Y., Ji, D.: Dispatched attention with multi-task learning for nested mention recognition. Inf. Sci. 513, 241–251 (2020)
Fuhr, N.: Some common mistakes in IR evaluation, and how they can be avoided. In: ACM SIGIR Forum, vol. 51, pp. 32–41. ACM New York, NY, USA (2018)
He, Z., Tao, C., Bian, J., Dumontier, M., Hogan, W.R.: Semantics-powered healthcare engineering and data analytics (2017)
Hilfiker, R., et al.: Exercise and other non-pharmaceutical interventions for cancer-related fatigue in patients during or after cancer treatment: a systematic review incorporating an indirect-comparisons meta-analysis. Br. J. Sports Med. 52(10), 651–658 (2018)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991 (2015)
Jaseena, K., David, J.M.: Issues, challenges, and solutions: big data mining. CS IT-CSCP 4(13), 131–140 (2014)
Jin, D., Szolovits, P.: PICO element detection in medical text via long short-term memory neural networks. In: Proceedings of the BioNLP 2018 workshop, pp. 67–75 (2018)
Joshi, A., Karimi, S., Sparks, R., Paris, C., MacIntyre, C.R.: A comparison of word-based and context-based representations for classification problems in health informatics. arXiv preprint arXiv:1906.05468 (2019)
Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., Moher, D.: Evidence summaries: the evolution of a rapid review approach. Syst. Rev. 1(1), 1–9 (2012)
Nye, B., et al.: A corpus with multi-level annotations of patients, interventions and outcomes to support language processing for medical literature. In: Proceedings of the conference. Association for Computational Linguistics. Meeting. vol. 2018, p. 197. NIH Public Access (2018)
Ruder, S.: An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098 (2017)
Russell, R., et al.: Systematic review methods. In: Issues and Challenges in Conducting Systematic Reviews to Support Development of Nutrient Reference Values: Workshop Summary Nutrition Research Series, vol. 2 (2009)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Xu, R., Garten, Y., Supekar, K.S., Das, A.K., Altman, R.B., Garber, A.M., et al.: Extracting subject demographic information from abstracts of randomized clinical trial reports. In: Medinfo 2007: Proceedings of the 12th World Congress on Health (Medical) Informatics; Building Sustainable Health Systems, p. 550. IOS Press (2007)
Zhang, T., Yu, Y., Mei, J., Tang, Z., Zhang, X., Li, S.: Unlocking the power of deep PICO extraction: Step-wise medical NER identification. arXiv preprint arXiv:2005.06601 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Dhrangadhariya, A., Aguilar, G., Solorio, T., Hilfiker, R., Müller, H. (2021). End-to-End Fine-Grained Neural Entity Recognition of Patients, Interventions, Outcomes. In: Candan, K.S., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2021. Lecture Notes in Computer Science(), vol 12880. Springer, Cham. https://doi.org/10.1007/978-3-030-85251-1_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-85251-1_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-85250-4
Online ISBN: 978-3-030-85251-1
eBook Packages: Computer ScienceComputer Science (R0)