Skip to main content
SpringerLink
Log in

Using R Markdown for Replicable Experiments in Evidence Based Medicine

  • Conference paper
  • First Online:
Book cover Experimental IR Meets Multilinguality, Multimodality, and Interaction (CLEF 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11018))

Abstract

In this paper, we propose a methodology based on the R Markdown framework for replicating an experiment of query rewriting in the context of medical eHealth. We present a study on how to re-propose the same task of systematic medical reviews with the same conditions and methodologies to a larger group of participants. The task is the CLEF eHealth Task Technologically Assisted Reviews in Empirical Medicine which consists in finding all the most relevant medical documents, given an information need, with the least effort. We study how lay people, students of a master degree in languages in this case, can help the retrieval system in finding more relevant documents by means of a query rewriting approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://goo.gl/CePVzY.

  2. 2.

    https://goo.gl/WgkqnZ.

  3. 3.

    http://www.centre-eval.org.

  4. 4.

    http://rmarkdown.rstudio.com.

  5. 5.

    https://github.com/gmdn/CLEF2018.

  6. 6.

    https://www.merriam-webster.com/medical.

  7. 7.

    http://www.medilexicon.com.

  8. 8.

    https://www.ncbi.nlm.nih.gov/pubmed/.

  9. 9.

    https://github.com/leifos/tar.

  10. 10.

    https://en.wikipedia.org/wiki/Pareto_efficiency.

References

  1. Berez-Kroeker, A.L., et al.: Reproducible research in linguistics: a position statement on data citation and attribution in our field. Linguistics 561(1), 1–18 (2017)

    Article  Google Scholar 

  2. Branco, A., Cohen, K.B., Vossen, P., Ide, N., Calzolari, N.: Replicability and reproducibility of research results for human language technology: introducing an LRE special section. Lang. Resour. Eval. 51(1), 1–5 (2017)

    Article  Google Scholar 

  3. Cohen, K.B., Xia, J., Roeder, C., Hunter, L.: Reproducibility in natural language processing: a case study of two R libraries for mining PubMed/MEDLINE. In: LREC 4REAL Workshop: Workshop on Research Results Reproducibility and Resources Citation in Science and Technology of Language, pp. 6–12. European Language Resources Association (ELRA) (2016)

    Google Scholar 

  4. Cormack, G.V., Grossman, M.R.: Scalability of continuous active learning for reliable high-recall text classification. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, CIKM 2016, pp. 1039–1048. ACM, New York (2016)

    Google Scholar 

  5. Cormack, G.V., Grossman, M.R.: Technology-assisted review in empirical medicine: waterloo participation in CLEF ehealth 2017. In: Working Notes of CLEF 2017 - Conference and Labs of the Evaluation Forum, Dublin, Ireland, 11–14 September 2017 (2017)

    Google Scholar 

  6. Cowie, A.P.: Phraseology: Theory, Analysis, and Applications. Oxford Studies in Lexicography and Lexicology. OUP Oxford, Oxford (1998)

    Google Scholar 

  7. Di Nunzio, G.M., Beghini, F., Vezzani, F., Henrot, G.: An interactive two-dimensional approach to query aspects rewriting in systematic reviews. IMS unipd at CLEF ehealth task 2. In: Working Notes of CLEF 2017 - Conference and Labs of the Evaluation Forum, Dublin, Ireland, 11–14 September 2017 (2017)

    Google Scholar 

  8. Di Nunzio, G.M., Beghini, F., Vezzani, F., Henrot, G.: A reproducible approach with R markdown to automatic classification of medical certificates in French. In: Proceedings of the Fourth Italian Conference on Computational Linguistics (CLiC-it 2017), Rome, Italy, 11–13 December 2017 (2017)

    Google Scholar 

  9. Ferro, N.: Reproducibility challenges in information retrieval evaluation. J. Data Inf. Qual. 8(2), 8:1–8:4 (2017)

    Google Scholar 

  10. Ferro, N., et al. (eds.): Advances in Information Retrieval. LNCS, vol. 9626. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-30671-1

    Book  Google Scholar 

  11. Gandrud, C.: Reproducible Research with R and R Studio, 2nd edn. Chapman and Hall/CRC, Boca Raton (2015)

    MATH  Google Scholar 

  12. Gouadec, D.: Terminologie: constitution des données. AFNOR gestion. AFNOR (1990)

    Google Scholar 

  13. Kanoulas, E., Li, D., Azzopardi, L., Spijker, R. (eds.) CLEF 2017 technologically assisted reviews in empirical medicine overview. In: Working Notes of CLEF 2017 - Conference and Labs of the Evaluation Forum, Dublin, Ireland, 11–14 September 2017, CEUR Workshop Proceedings. CEUR-WS.org (2017)

    Google Scholar 

  14. Karimi, S., Pohl, S., Scholer, F., Cavedon, L., Zobel, J.: Boolean versus ranked querying for biomedical systematic reviews. BMC Med. Inform. Decis. Mak. 10, 58–58 (2010)

    Article  Google Scholar 

  15. Knuth, D.E.: Literate programming. Comput. J. 27(2), 97–111 (1984)

    Article  Google Scholar 

  16. L’Homme, M.-C.: Sur la notion de “terme”. Meta 50(4), 1112–1132 (2005)

    Article  Google Scholar 

  17. Liberman, M.: Validation of results in linguistic science and technology: terminology, problems, and solutions. In: Branco, A., Calzolari, N., Choukri, K. (eds.) Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Paris, France, May 2018. European Language Resources Association (ELRA) (2018)

    Google Scholar 

  18. Martín, A.S., L’Homme, M.-C.: Definition patterns for predicative terms in specialized lexical resources. In: Proceedings of the Ninth International Conference on Language Resources and Evaluation, LREC 2014, Reykjavik, Iceland, 26–31 May 2014, pp. 3748–3755 (2014)

    Google Scholar 

  19. Miwa, M., Thomas, J., O’Mara-Eves, A., Ananiadou, S.: Reducing systematic review workload through certainty-based screening. J. Biomed. Inform. 51, 242–253 (2014)

    Article  Google Scholar 

  20. Di Nunzio, G.M.: A study of an automatic stopping strategy for technologically assisted medical reviews. In: Pasi, G., Piwowarski, B., Azzopardi, L., Hanbury, A. (eds.) ECIR 2018. LNCS, vol. 10772, pp. 672–677. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-76941-7_61

    Chapter  Google Scholar 

  21. Rastier, F.: Sémantique interprétative. Formes sémiotiques. Presses universitaires de France, Paris (1987)

    Google Scholar 

  22. Robertson, S.E., Walker, S., Jones, S., Hancock-Beaulieu, M., Gatford, M.: Okapi at TREC-3. In: Proceedings of the Third Text REtrieval Conference, TREC 1994, Gaithersburg, Maryland, USA, 2–4 November 1994, pp. 109–126 (1994)

    Google Scholar 

  23. Singh, G., Thomas, J., Shawe-Taylor, J.: Improving active learning in systematic reviews. CoRR, abs/1801.09496 (2018)

    Google Scholar 

  24. Vezzani, F., Di Nunzio, G.M., Henrot, G.: Trimed: a multilingual terminological database. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC 2018, Miyazaky, Japan, 7–12 May 2018 (2018, in press)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Engineering, University of Padua, Padova, Italy

    Giorgio Maria Di Nunzio

  2. Department of Linguistic and Literary Studies, University of Padua, Padova, Italy

    Federica Vezzani

Authors
  1. Giorgio Maria Di Nunzio
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Federica Vezzani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Federica Vezzani .

Editor information

Editors and Affiliations

  1. Aix-Marseille University, Marseille Cedex 20, France

    Patrice Bellot

  2. Virtual University of Tunis, Tunis, Tunisia

    Chiraz Trabelsi

  3. Systèmes d’informations, Big Data et Rec, Institut de Recherche en Informatique de, Toulouse Cedex 04, France

    Josiane Mothe

  4. Department of Computer Science, University of Huddersfield, Huddersfield, United Kingdom

    Fionn Murtagh

  5. DIRO, Universite de Montreal, Montreal, Québec, Canada

    Jian Yun Nie

  6. Pierre and Marie Curie University, Paris Cedex 05, France

    Laure Soulier

  7. Université d'Avignon et des Pays de, Avignon, France

    Eric SanJuan

  8. Department of Information Engineering, University of Padua, Padua, Padova, Italy

    Linda Cappellato

  9. University of Padua, Padua, Italy

    Nicola Ferro

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Di Nunzio, G.M., Vezzani, F. (2018). Using R Markdown for Replicable Experiments in Evidence Based Medicine. In: Bellot, P., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2018. Lecture Notes in Computer Science(), vol 11018. Springer, Cham. https://doi.org/10.1007/978-3-319-98932-7_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-98932-7_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-98931-0

  • Online ISBN: 978-3-319-98932-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation