Abstract
Systematic medical reviews are a method to collect the findings from multiple studies in a reliable way. Given budget and time constraints, limiting the recall of a search may undermine the quality of a review to such a degree that the validity of its findings is questionable. In this paper, we investigate a variable threshold approach to tackle the problem of a total recall task in medical reviews proposed by a Cross-Language Evaluation Forum (CLEF) eHealth lab in 2017. Compared to the official results submitted to the CLEF eHealth task, our approach performed consistently better over all the range of thresholds considered achieving a recall greater than 0.95 with 25,000 documents less than the best performing systems. The runs and the source code to generate the analyses of this paper are available at the following GitHub repository (https://github.com/gmdn/ECIR2018).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Cochrane Handbook for Systematic Reviews of Interventions http://handbook-5-1.cochrane.org.
- 2.
- 3.
References
Di Nunzio, G.M.: A new decision to take for cost-sensitive Naïve Bayes classifiers. Inf. Process. Manage. 50(5), 653–674 (2014)
Di Nunzio, G.M.: Interactive text categorisation: the geometry of likelihood spaces. In: Lai, C., Giuliani, A., Semeraro, G. (eds.) Information Filtering and Retrieval. SCI, vol. 668, pp. 13–34. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-46135-9_2
Goeuriot, L., et al.: CLEF 2017 eHealth evaluation lab overview. In: Jones, G.J.F., et al. (eds.) CLEF 2017. LNCS, vol. 10456, pp. 291–303. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65813-1_26
Grossman, M.R., Cormack, G.V., Roegiest, A.: TREC 2016 total recall track overview. In: Proceedings of the Twenty-Fifth TREC 2016, Gaithersburg, Maryland, USA, 15–18 November (2016)
Kanoulas, E., Li, D., Azzopardi, L., Spijker, R.: CLEF 2017 technologically assisted reviews in empirical medicine overview. In: Working Notes of CLEF 2017 - Conference and Labs of the Evaluation Forum, CEUR Workshop Proceedings, Dublin, Ireland, 11–14 September 2017. CEUR-WS.org (2017)
Di Nunzio, G.M., Beghini, F., Vezzani, F., Henrot, G.: An interactive two-dimensional approach to query aspects rewriting in systematic reviews. IMS Unipd At CLEF eHealth Task 2. In: Working Notes of CLEF 2017 - Conference and Labs of the Evaluation Forum, Dublin, Ireland, 11–14 September (2017)
O’Mara-Eves, A., Thomas, J., McNaught, J., Miwa, M., Ananiadou, S.: Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst. Rev. 4(1), 5 (2015)
Robertson, S.E., Zaragoza, H.: The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retr. 3(4), 333–389 (2009)
Roegiest, A., Cormack, G.V., Grossman, M.R., Clarke, C.L.A.: TREC 2015 total recall track overview. In: Proceedings of the Twenty-fourth TREC 2015, Gaithersburg, Maryland, USA, 17–20 November (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Di Nunzio, G.M. (2018). A Study of an Automatic Stopping Strategy for Technologically Assisted Medical Reviews. In: Pasi, G., Piwowarski, B., Azzopardi, L., Hanbury, A. (eds) Advances in Information Retrieval. ECIR 2018. Lecture Notes in Computer Science(), vol 10772. Springer, Cham. https://doi.org/10.1007/978-3-319-76941-7_61
Download citation
DOI: https://doi.org/10.1007/978-3-319-76941-7_61
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-76940-0
Online ISBN: 978-3-319-76941-7
eBook Packages: Computer ScienceComputer Science (R0)