Skip to main content

Advertisement

Springer Nature Link
Log in

Achieving High Research Reporting Quality Through the Use of Computational Ontologies

  • Published:
Neuroinformatics Aims and scope Submit manuscript

Abstract

Systematic reviews and meta-analyses constitute one of the central pillars of evidence-based medicine. However, clinical trials are poorly reported which delays meta-analyses and consequently the translation of clinical research findings to clinical practice. We propose a Center of Excellence in Research Reporting in Neurosurgery (CERR-N) and the creation of a clinically significant computational ontology to encode Randomized Controlled Trials (RCT) studies in neurosurgery. A 128 element strong computational ontology was derived from the Trial Bank ontology by omitting classes which were not required to perform meta-analysis. Three researchers from our team tagged five randomly selected RCT’s each, published in the last 5 years (2004–2008), in the Journal of Neurosurgery (JoN), Neurosurgery Journal (NJ) and Journal of Neurotrauma (JoNT). We evaluated inter and intra observer reliability for the ontology using percent agreement and kappa coefficient. The inter-observer agreement was 76.4%, 75.97% and 74.9% and intra-observer agreement was 89.8%, 80.8% and 86.56% for JoN, NJ and JoNT respectively. The inter-observer kappa coefficient was 0.60, 0.54 and 0.53 and the intra-observer kappa coefficient was 0.79, 0.82 and 0.79 for JoN, NJ and JoNT journals respectively. The high degree of inter and intra-observer agreement confirms tagging consistency in sections of a given scientific manuscript. Standardizing reporting for neurosurgery articles can be reliably achieved through the integration of a computational ontology within the context of a CERR-N. This approach holds potential for the overall improvement in the quality of reporting of RCTs in neurosurgery, ultimately streamlining the translation of clinical research findings to improvement in patient care.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., et al. (2001). The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Annals of Internal Medicine, 134(8), 663–694.

    PubMed  CAS  Google Scholar 

  • Andersen, J., Larsen, J. K., Schultz, V., Nielsen, B. M., Kørner, A., Behnke, K., et al. (1989). The brief psychiatric rating scale. Dimension of schizophrenia—reliability and construct validity. Psychopathology, 22(2–3), 168–176.

    Article  PubMed  CAS  Google Scholar 

  • Biesen, W. V., Verbeke, F., & Vanholder, R. (2008). An infallible recipe? A story of cinnamon, soufflé and meta-analysis. Nephrology, Dialysis, Transplantation, 23(9), 2729–2732.

    Article  PubMed  Google Scholar 

  • Bossuyt, P. (2004). The quality of reporting in diagnostic test research: getting better, still not optimal. Clinical Chemistry, 50, 465–466.

    Article  PubMed  CAS  Google Scholar 

  • Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., et al. (2004). Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Family Practice, 21, 4–10.

    Article  PubMed  Google Scholar 

  • Buttram, S., Wisniewski, S., Jackson, E., Adelson, D., Feldman, K., Bayir, H., et al. (2007). Multiplex assessment of cytokine and chemokine levels in cerebrospinal fluid following severe pediatric traumatic brain injury: effects of moderate hypothermia. Journal of Neurotrauma, 24(11), 1707–1718. doi:10.1089/neu.2007.0349.

    Article  PubMed  Google Scholar 

  • Choi, G., Raiturker, P. P., Kim, M. J., Chung, D. J., Chae, Y. S., & Lee, S. H. (2005). The effect of early isolated lumbar extension exercise program for patients with herniated disc undergoing lumbar discectomy. Neurosurgery, 57, 764–772.

    Article  PubMed  Google Scholar 

  • Clarke, M. J., & Stewart, L. A. (1994). Obtaining data from randomised controlled trials: how much do we need for reliable and informative meta-analyses? BMJ, 309(6960), 1007–1010.

    PubMed  CAS  Google Scholar 

  • Cook, D. J., Greengold, N. L., Ellrodt, A. G., & Weingarten, S. R. (1997). The relation between systematic reviews and practice guidelines. Annals of Internal Medicine, 127(3), 210–216.

    PubMed  CAS  Google Scholar 

  • Cook, C., Hannley, M., Richardson, J. K., Michon, J., Harker, M., & Pietrobon, R. (2007). Real-time updates of meta-analyses of HIV treatments supported by a biomedical ontology. Accountability in Research, 14(1), 1–18.

    Article  PubMed  Google Scholar 

  • Crippa, J. A. S., Sanches, R. F., Hallak, J. E. C., Loureiro, S. R., & Zuardi, A. W. (2002). Factor structure of Bech’s version of the brief psychiatric rating scale in Brazilian patients. Brazilian Journal of Medical and Biological Research, 35(10), 1209–1213.

    Article  PubMed  CAS  Google Scholar 

  • Fergusen, S., & Macdonald, R. L. (2007). Predictors of cerebral infarction in patients with aneurysmal subarachnoid hemorrhage. Neurosurgery, 60, 658–667.

    Article  PubMed  Google Scholar 

  • Fonseca, F., & Martin, J. (2007). Learning the differences between ontologies and conceptual schemas through ontology-driven information systems. JAIS - Journal of Association for Information Systems - Special Issue on Ontologies in the Context of IS, 8(2), 129–142.

    Google Scholar 

  • Fung, A. E., Palanki, R., Bakri, S. J., Depperschmidt, E., & Gibson, A. (2009). Applying the CONSORT and STROBE statements to evaluate the reporting quality of neovascular age-related macular degeneration studies. Ophthalmology, 116(2), 286–296.

    Article  PubMed  Google Scholar 

  • Groves, T. (2008). Enhancing the quality and transparency of health research. QBMJ, 337, a718.

    Article  Google Scholar 

  • Gruber, T. (2008). Ontology. Entry in the Encyclopedia of Database Systems. Ling Liu and M. Tamer Özsu (Eds.), Springer-Verlag, to appear in 2008. Provides a definition of ontology as a technical term for computer science, tracing its historical context from philosophy and AI.

  • Gruber, T. R. (1993). Toward principles for the design of ontologies used for knowledge sharing.

  • Haines, S. J. (1983). Randomized clinical trials in neurosurgery. Neurosurgery, 12(3), 259–264.

    Article  PubMed  CAS  Google Scholar 

  • Han, C., Kwak, K. P., Marks, D. M., Pae, C. U., Wu, L. T., Bhatia, K. S., Masand, P. S., Patkar, A. A., et al. (2009). The impact of the CONSORT statement on reporting of randomized clinical trials in psychiatry. Contemporary Clinical Trials, 30(2), 116–122.

    Article  PubMed  Google Scholar 

  • Hopewell, S., Altman, D. G., Moher, D., & Schulz, K. F. (2008). Endorsement of the CONSORT statement by high impact factor medical journals: a survey of journal editors and journal ‘Instructions to Authors’. Trials, 9, 20.

    Article  PubMed  Google Scholar 

  • Hripcsak, G., & Heitjan, D. (2002). Measuring agreement in medical informatics reliability studies. Journal of Biomedical Informatics Archive, 35(2), 99–110.

    Article  Google Scholar 

  • Jarlais, D. C., Lyles, C., Crepaz, N., & TREND Group. (2004). Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement. Public Health, 94, 361–366.

    Article  Google Scholar 

  • Jiang, J. Y., Wei, X. U., Li, W. P., Xu, W. H., Jun, Z., Bao, Y. H., et al. (2005). Efficacy of standard trauma craniectomy for refractory intracranial hypertension with severe traumatic brain injury: a multicenter, prospective, randomized controlled study. Journal of Neurotrauma, 22(6), 623–628.

    Article  PubMed  Google Scholar 

  • Kane, R. L., Wang, J., & Garrard, J. (2007). Reporting in randomized clinical trials improved after adoption of the CONSORT statement. Journal of Clinical Epidemiology, 60(3), 241–249.

    Article  PubMed  Google Scholar 

  • Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.

    Article  PubMed  CAS  Google Scholar 

  • Manchikanti, L., Derby, R., Wolfer, L., Singh, V., Datta, S., & Hirsch, J. A. (2008). Evidence based medicine, systematic reviews, and guidelines in interventional pain management, part I: introduction and general considerations. Pain Physician, 11(2), 161–186.

    PubMed  Google Scholar 

  • Manser, R., & Walters, E. H. (2001). What is evidence-based medicine and the role of the systematic review: the revolution coming your way. Monaldi Archives for Chest Diseases, 56(1), 33–38.

    CAS  Google Scholar 

  • Marmarou, A., Guy, M., Murphey, L., Roy, F., Layani, L., Combal, J. P., et al. (2005). A single dose, three-arm, placebo-controlled, phase I study of the bradykinin B2 receptor antagonist Anatibant (LF16-0687Ms) in patients with severe traumatic brain injury. Journal of Neurotrauma, 22(12), 1444–1455.

    Article  PubMed  Google Scholar 

  • Mazzeo, A. T., Kunene, N. K., Gilman, C. B., Hamm, R. J., Hafez, N., & Bullock, M. R. (2006). Severe human traumatic brain injury, but not cyclosporin a treatment, depresses activated T lymphocytes early after injury. Journal of Neurotrauma, 23(6), 962–975.

    Article  PubMed  Google Scholar 

  • Meinert, C. L. (1998). Beyond CONSORT: need for improved reporting standards for clinical trials. Consolidated standards of reporting trials. JAMA, 279(18), 1487–1489.

    Article  PubMed  CAS  Google Scholar 

  • Meinert, C. L., Tonascia, S., & Higgins, K. (1984). Content of reports on clinical trials: a critical review. Controlled Clinical Trials, 5(4), 328–347.

    Article  PubMed  CAS  Google Scholar 

  • Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., et al. (1998). Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet, 352(9128), 609–613.

    Article  PubMed  CAS  Google Scholar 

  • Moher, D., Cook, D., Olkin, I., Stroup, D. F., & for the QUOROM Group. (2000). Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. QUOROM Group. The British Journal of Surgery, 87(11), 1448–1454.

    Article  PubMed  CAS  Google Scholar 

  • Moher, D., Schulz, K. F., & Altman, D. G. (2001a). The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet, 357(9263), 1191–1194.

    Article  CAS  Google Scholar 

  • Moher, D., Jones, A., Lepage, L., & CONSORT Group (Consolidated Standards for Reporting of Trials). (2001b). Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. JAMA, 285(15), 1992.

    Article  CAS  Google Scholar 

  • Moher, D., Simera, I., Schulz, K. F., Hoey, J., & Altman, D. G. (2008). Helping editors, peer reviewers and authors improve the clarity, completeness and transparency of reporting health research. BMC Medicine, 6, 13.

    Article  PubMed  Google Scholar 

  • Mummaneni, P. V., Burkus, J. K., Haid, R. W., Traynelis, V. C., & Zdeblick, T. A. (2007). Clinical and radiographic analysis of cervical disc arthroplasty compared with allograft fusion: a randomized controlled clinical trial. Neurosurgery Spine, 6, 198–209.

    Article  Google Scholar 

  • Naff, N. J., Hanley, D. F., Keyl, P. M., Tuhrim, S., Kraut, M., Bederson, J., et al. (2004). Intraventricular thrombolysis speeds blood clot resolution: results of a pilot, prospective, randomized, double-blind, controlled trial. Neurosurgery, 54, 577–584.

    Article  PubMed  Google Scholar 

  • North, R. B., Kidd, D. H., Farrokhi, F., & Piantadosi, S. A. (2005). Spinal cord stimulation versus repeated lumbosacral spine surgery for chronic pain: a randomized, controlled trial. Neurosurgery, 56, 98–107.

    PubMed  Google Scholar 

  • Noy, N. F., & McGuinness, D. L. (2001). Ontology Development 101: A guide to creating your first ontology. Stanford Knowledge Systems Laboratory Technical Report KSL-01-05 and Stanford Medical Informatics Technical Report. SMI-2001-0880.

  • Petersson, H., Gill, H., & Ahlfeldt, H. (2002). A variance-based measure of inter-rater agreement in medical databases. Journal of Biomedical Informatics, 35(5–6), 331–342.

    Article  PubMed  Google Scholar 

  • Phadtare, A., Bahmani, A., Shah, A., & Pietrobon, R. (2009). Scientific writing: a randomized controlled trial comparing standard and on-line instruction. BMC Medical Education, 9, 27.

    Article  PubMed  Google Scholar 

  • Pocock, S. J., Hughes, M. D., & Lee, R. J. (1987). Statistical problems in the reporting of clinical trials. A survey of three medical journals. The New England Journal of Medicine, 317(7), 426–432.

    Article  PubMed  CAS  Google Scholar 

  • Racer Pro 2.0 available at: http://www.racer-systems.com/index.phtml.

  • Righesso, O., Falavigna, A., & Avanzi, O. (2007). Comparison of open discectomy with microendoscopic discectomy in lumbar disc herniations: results of a randomized controlled trial. Neurosurgery, 61, 545–549.

    Article  PubMed  Google Scholar 

  • Rogers, J. E. (2006). Quality assurance of medical ontologies. Methods of Information in Medicine, 45(3), 267–274.

    PubMed  CAS  Google Scholar 

  • Sackett, D. (1996). Evidence based medicine: what it is and what it isn’t. BMJ, 312, 71–72.

    PubMed  CAS  Google Scholar 

  • Schulz, E., Barrett, J. W., & Colin, P. (1998). Read code quality assurance from simple syntax to semantic stability. Journal of the American Medical Informatics Association, 5, 337–346.

    PubMed  CAS  Google Scholar 

  • Shankar, R. D., Martins, S. B., O’Connor, M., Parrish, D. B., & Das, A. K. (2007). An ontology-based architecture for integration of clinical trials management applications. AMIA Annu Symp Proc. 661–665.

  • Sim, I., & Detmer, DE. (2005). Beyond trial registration: a global trial bank for clinical trial reporting. PLoS Medicine, 2(11), e365.

    Article  PubMed  Google Scholar 

  • Sim, I., Owens, D. K., Lavori, P. W., & Rennels, G. D. (2000). Electronic trial banks: a complementary method for reporting randomized trials. Medical Decision Making, 20(4), 440–450.

    Article  PubMed  CAS  Google Scholar 

  • Sim, I., Gorman, P., Greenes, R. A., Haynes, R. B., Kaplan, B., Lehmann, H., et al. (2001). Clinical decision support systems for the practice of evidence-based medicine. Journal of the American Medical Informatics Association, 8(6), 527–534.

    PubMed  CAS  Google Scholar 

  • Sim, I., Carini, S., Olasov, B., & Jeng, S. (2004). An ontology of randomized controlled trials for evidence-based practice: content specification and evaluation using the competency decomposition method. Journal of Biomedical Informatics, 37(2), 108–119.

    Article  PubMed  Google Scholar 

  • Stroup, D. F., Berlin, J. A., Morton, S. C., Olkin, I., Williamson, G. D., Rennie, D., et al. (2000). Meta-analysis of observational studies in epidemiology a proposal for reporting. JAMA, 283, 2008–2012.

    Article  PubMed  CAS  Google Scholar 

  • Tapia-Perez, J. H., Sanchez-Aguilar, M., Torres-Corzo, J. G., Gordillo-Moscoso, A., Martinez-Perez, P., Madeville, P., et al. (2008). Effect of rosuvastatin on amnesia and disorientation after traumatic brain injury (NCT003229758). Journal of Neurotrauma, 25, 1011–1017.

    Article  PubMed  Google Scholar 

  • Tendal, B., Higgins, J. P., Jüni, P., Hrobjartsson, A., Trelle, S., Nüesch, E. et al. (2008). Observer variation when extracting data for the calculation of a standardized mean difference, Poster presentation at 16th Cochrane Colloquium, Freiburg, 3-7 October 2008.

  • Tseng, M. Y., Hutchinson, P. J., Turner, C. L., Czosnyka, M., Richards, H., Pickard, J. D., et al. (2007). Biological effects of acute pravastatin treatment in patients after aneurysmal subarachnoid hemorrhage: a double-blind, placebo-controlled trial. Journal of Neurosurgery, 107, 1092–1100.

    Article  PubMed  CAS  Google Scholar 

  • Vajkoczy, P., Meyer, B., Weidauer, S., Raabe, A., Thome, C., Ringel, F., et al. (2005). Clazosentan (AXV-034343), a selective endothelin A receptor antagonist, in the prevention of cerebral vasospasm following severe aneurysmal subarachnoid hemorrhage: results of a randomized, double-blind, placebo-controlled, multicenter Phase IIa study. Journal of Neurosurgery, 103, 9–17.

    Article  PubMed  CAS  Google Scholar 

  • Vranos, G., Tatsioni, A., Polyzoidis, K., Ioannidis, J. P. A., et al. (2004). Randomized trials of neurosurgical interventions: A systematic appraisal. Neurosurgery, 55(1), 18–26.

    Article  PubMed  Google Scholar 

  • Wilczynski, N. L. (2008). Quality of reporting of diagnostic accuracy studies: no change since STARD statement publication—before-and-after study. Radiology, 248(3), 817–823.

    Article  PubMed  Google Scholar 

  • Willems, P. W., Taphoorn, M. J., Burger, H., Berkelbach van der Sprenkel, J. W., & Tulleken, C. A. (2006). Effectiveness of neuronavigation in resecting solitary intracerebral contrast-enhancing tumors: a randomized controlled trial. Journal of Neurosurgery, 104, 360–368.

    Article  PubMed  Google Scholar 

  • Xu, L., Li, J., Zhang, M., Ai, C., & Wang, L. (2008). Chinese authors do need CONSORT: reporting quality assessment for five leading Chinese medical journals. Contemporary Clinical Trials, 29(5), 727–731.

    Article  PubMed  CAS  Google Scholar 

Download references

Acknowledgments

We would like to thank the Research on Research group (www.researchonresearch.org), Duke University, USA for manuscript writing templates. (Phadtare et al. 2009). We would also like to thank FS from the Cochrane group who helped us modify the ontology, by giving us input as to which classes would be required for a meta-analysis.

Author information

Authors and Affiliations

  1. National Neuroscience Institute, Singapore, Singapore

    Amrapali Zaveri & Beng Ti Ang

  2. Duke-NUS Graduate Medical School, Singapore, Singapore

    Amrapali Zaveri, Jatin Shah, Shreyasee Pradhan & Ricardo Pietrobon

  3. Australasian Cochrane Centre, Singapore, Singapore

    Edwin Chan

  4. INSERM U936, Universite de Rennes1, Rennes, France

    Olivier Dameron

  5. Research on Research Group, Sao Paulo, Brazil

    Luciana Cofiel

Authors
  1. Amrapali Zaveri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luciana Cofiel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jatin Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shreyasee Pradhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Edwin Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Olivier Dameron
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ricardo Pietrobon
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Beng Ti Ang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amrapali Zaveri.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zaveri, A., Cofiel, L., Shah, J. et al. Achieving High Research Reporting Quality Through the Use of Computational Ontologies. Neuroinform 8, 261–271 (2010). https://doi.org/10.1007/s12021-010-9079-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12021-010-9079-5

Keywords

Search

Navigation