Abstract
In this paper, we propose a methodology based on the R Markdown framework for replicating an experiment of query rewriting in the context of medical eHealth. We present a study on how to re-propose the same task of systematic medical reviews with the same conditions and methodologies to a larger group of participants. The task is the CLEF eHealth Task Technologically Assisted Reviews in Empirical Medicine which consists in finding all the most relevant medical documents, given an information need, with the least effort. We study how lay people, students of a master degree in languages in this case, can help the retrieval system in finding more relevant documents by means of a query rewriting approach.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
References
Berez-Kroeker, A.L., et al.: Reproducible research in linguistics: a position statement on data citation and attribution in our field. Linguistics 561(1), 1–18 (2017)
Branco, A., Cohen, K.B., Vossen, P., Ide, N., Calzolari, N.: Replicability and reproducibility of research results for human language technology: introducing an LRE special section. Lang. Resour. Eval. 51(1), 1–5 (2017)
Cohen, K.B., Xia, J., Roeder, C., Hunter, L.: Reproducibility in natural language processing: a case study of two R libraries for mining PubMed/MEDLINE. In: LREC 4REAL Workshop: Workshop on Research Results Reproducibility and Resources Citation in Science and Technology of Language, pp. 6–12. European Language Resources Association (ELRA) (2016)
Cormack, G.V., Grossman, M.R.: Scalability of continuous active learning for reliable high-recall text classification. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, CIKM 2016, pp. 1039–1048. ACM, New York (2016)
Cormack, G.V., Grossman, M.R.: Technology-assisted review in empirical medicine: waterloo participation in CLEF ehealth 2017. In: Working Notes of CLEF 2017 - Conference and Labs of the Evaluation Forum, Dublin, Ireland, 11–14 September 2017 (2017)
Cowie, A.P.: Phraseology: Theory, Analysis, and Applications. Oxford Studies in Lexicography and Lexicology. OUP Oxford, Oxford (1998)
Di Nunzio, G.M., Beghini, F., Vezzani, F., Henrot, G.: An interactive two-dimensional approach to query aspects rewriting in systematic reviews. IMS unipd at CLEF ehealth task 2. In: Working Notes of CLEF 2017 - Conference and Labs of the Evaluation Forum, Dublin, Ireland, 11–14 September 2017 (2017)
Di Nunzio, G.M., Beghini, F., Vezzani, F., Henrot, G.: A reproducible approach with R markdown to automatic classification of medical certificates in French. In: Proceedings of the Fourth Italian Conference on Computational Linguistics (CLiC-it 2017), Rome, Italy, 11–13 December 2017 (2017)
Ferro, N.: Reproducibility challenges in information retrieval evaluation. J. Data Inf. Qual. 8(2), 8:1–8:4 (2017)
Ferro, N., et al. (eds.): Advances in Information Retrieval. LNCS, vol. 9626. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-30671-1
Gandrud, C.: Reproducible Research with R and R Studio, 2nd edn. Chapman and Hall/CRC, Boca Raton (2015)
Gouadec, D.: Terminologie: constitution des données. AFNOR gestion. AFNOR (1990)
Kanoulas, E., Li, D., Azzopardi, L., Spijker, R. (eds.) CLEF 2017 technologically assisted reviews in empirical medicine overview. In: Working Notes of CLEF 2017 - Conference and Labs of the Evaluation Forum, Dublin, Ireland, 11–14 September 2017, CEUR Workshop Proceedings. CEUR-WS.org (2017)
Karimi, S., Pohl, S., Scholer, F., Cavedon, L., Zobel, J.: Boolean versus ranked querying for biomedical systematic reviews. BMC Med. Inform. Decis. Mak. 10, 58–58 (2010)
Knuth, D.E.: Literate programming. Comput. J. 27(2), 97–111 (1984)
L’Homme, M.-C.: Sur la notion de “terme”. Meta 50(4), 1112–1132 (2005)
Liberman, M.: Validation of results in linguistic science and technology: terminology, problems, and solutions. In: Branco, A., Calzolari, N., Choukri, K. (eds.) Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Paris, France, May 2018. European Language Resources Association (ELRA) (2018)
Martín, A.S., L’Homme, M.-C.: Definition patterns for predicative terms in specialized lexical resources. In: Proceedings of the Ninth International Conference on Language Resources and Evaluation, LREC 2014, Reykjavik, Iceland, 26–31 May 2014, pp. 3748–3755 (2014)
Miwa, M., Thomas, J., O’Mara-Eves, A., Ananiadou, S.: Reducing systematic review workload through certainty-based screening. J. Biomed. Inform. 51, 242–253 (2014)
Di Nunzio, G.M.: A study of an automatic stopping strategy for technologically assisted medical reviews. In: Pasi, G., Piwowarski, B., Azzopardi, L., Hanbury, A. (eds.) ECIR 2018. LNCS, vol. 10772, pp. 672–677. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-76941-7_61
Rastier, F.: Sémantique interprétative. Formes sémiotiques. Presses universitaires de France, Paris (1987)
Robertson, S.E., Walker, S., Jones, S., Hancock-Beaulieu, M., Gatford, M.: Okapi at TREC-3. In: Proceedings of the Third Text REtrieval Conference, TREC 1994, Gaithersburg, Maryland, USA, 2–4 November 1994, pp. 109–126 (1994)
Singh, G., Thomas, J., Shawe-Taylor, J.: Improving active learning in systematic reviews. CoRR, abs/1801.09496 (2018)
Vezzani, F., Di Nunzio, G.M., Henrot, G.: Trimed: a multilingual terminological database. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC 2018, Miyazaky, Japan, 7–12 May 2018 (2018, in press)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Di Nunzio, G.M., Vezzani, F. (2018). Using R Markdown for Replicable Experiments in Evidence Based Medicine. In: Bellot, P., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2018. Lecture Notes in Computer Science(), vol 11018. Springer, Cham. https://doi.org/10.1007/978-3-319-98932-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-98932-7_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-98931-0
Online ISBN: 978-3-319-98932-7
eBook Packages: Computer ScienceComputer Science (R0)