Development of a tool for comprehensive evaluation of population-based cancer registries

https://doi.org/10.1016/j.ijmedinf.2018.06.006Get rights and content

Highlights

  • Population-based cancer registries should be evaluated comprehensively with regards to their inputs, processes and outputs.

  • There are few comprehensive tools for comprehensive evaluation of population-based cancer registries, especially in low- and middle-income countries.

  • A comprehensive evaluation tool was developed for population-based cancer registry and can be tailored for other countries.

  • This tool is used to evaluate guidelines, analysis and reports, governance and infrastructure, information technology, personnel knowledge, practice and data quality.

Abstract

Objective

Several methods have been suggested for evaluation of population-based cancer registries (PBCR) worldwide. However, most of these methods evaluate the data and outputs of the cancer registries. This study aimed to develop a comprehensive tool and protocol for evaluation of inputs, processes and outputs of a PBCR.

Methods

The standards of the North American Association of Central Cancer Registries (NAACCR) were used to draft a comprehensive checklist. In addition, the national guidelines of PBCR were used to develop a questionnaire for evaluation of knowledge and practice of the PBCR personnel. Furthermore, a protocol for evaluation of the completeness and validity of the PBCR data was developed according to the International Agency for Research on Cancer (IARC) and the NAACCR guidelines. A 0–4 Likert based score and expert opinions (10 experts) were used to assess validity of the eight questionnaires/checklists. A modified Delphi method was applied to validate the checklists and questionnaires. Questions with a score higher than 3 remained in the final tools.

Results

The final package consists of 546 questions including 108 (19.8%) for evaluation of guidelines, 54 (9.9%) for analysis and reports, 87 (15.9%) for governance and infrastructure, 155 (28.4%) for information technology, 21 (3.8%) for personnel knowledge and 121 (22.2%) for their practice. Additionally, data quality indicators were also considered for evaluation of PBCRs.

Conclusion

This comprehensive tool can be used to show the gaps and limitations of the PBCR programs and provide informative clues for their improvement.

Introduction

According to Globocan, in 2012, 14.1 million new cancer cases and also 8.2 million cancer-related deaths occurred. Out of these statistics, 57% of new cancer cases (8 million) and 65% of cancer-related deaths occurred in less developed countries [1]. In addition, the incidence rate of cancers is rapidly increasing in low and middle-income countries (LMCs) [[2], [3], [4]].

The population-based cancer registry (PBCR) is a key component of any cancer control program [5]. PBCR programs continuously collect, manage, analyze and report valid and reliable data of cancer incidence and mortality in a predefined population from multiple sources [3,[5], [6], [7]]. The information obtained from PBCRs is important for planning and evaluating any cancer control and cancer surveillance program, as well as measuring a variety of indicators such as incidence, mortality and survival rates in a population [[8], [9], [10]].

Establishment of PBCRs is increasing worldwide, especially in developed countries; however, several low and middle-income countries have not established high quality PBCRs yet. In some countries, particularly in Asia and Africa, PBCRs cover a small percentage of the population [3]. The quality of cancer registration is crucial to ensure that a PBCR works properly and produces high quality and reliable results [[11], [12], [13]]. It is clearly useful to evaluate the methods and processes of a PBCR, to identify the gaps in its functions in producing accurate data and developing improvement plans. Therefore, comprehensive evaluation of the program considering the inputs, processes and outputs may help in identifying the gaps and improving the program [[14], [15], [16]].

Many researchers have developed numerous criteria or indicators to evaluate PBCR programs. Parkin [14] and Bray [5] emphasized on data quality and suggested four dimensions to evaluate the data quality of a cancer registry program, these include completeness, timeliness, validity and comparability; all are related to the output of the PBCRs. Several groups have evaluated the quality of cancer registry data based on this framework [12,[17], [18], [19], [20], [21], [22], [23]]. In addition, the International Agency for Research on Cancer (IARC), a specialized cancer agency of the World Health Organization, presented a monograph called Cancer Incidence in Five Continents (CI5) in which reports from different cancer registries in the world are published. They have an evaluation system focused on data quality. In this system, after receiving data from different cancer registries, the CI5 editors check the data based on the IARC-CHECK programs. Then, they may be accepted for publication [24,25].

Additional criteria and guidelines for assessment of the inputs and processes of a PBCR have also been considered. In North America, the successful PBCR programs are continuously evaluated based on an accreditation process [26]. In addition, the European Network of Cancer Registries (ENCR) has developed a process called Structured Reviews of Cancer Registries for evaluation of the European cancer registries [27,28]. Navarro et al. [29] also suggested a process for external evaluation of PBCRs in Spain and Latin American countries. They found that the successful PBCRs that published their data in the CI5 achieved high scores in the external evaluation [29]. The results indicate that their external evaluation tool was appropirate and had a high sensitivity and positive predictive value. In other words, results of external evaluation would be good indicator of whether the registry will achieve the objectives and pass the evaluation criteria of IARC monograph group and be published in the CI5.

All these tools are restricted to some aspects of the cancer registry and no comprehensive evaluation program was found. In addition, the available tools should be tailored to the requirements of the PBCR in each country. Therefore, there is a need to develop a comprehensive evaluation program for PBCRs. In this regard, standards provided by the North American Association of Central Cancer Registries (NAACCR) and the IARC were used and a comprehensive package of checklists and questionnaires was developed for evaluation of PBCRs. Use of this evaluation package will provide informative clues regarding gaps and areas for interventions and improvement of PBCR.

Section snippets

Cancer registration in I.R of Iran

The history and situation of the cancer registry in I.R of Iran has been published previously [4,[30], [31], [32]]. In brief, a national pathology-based cancer registry program was established by the Ministry of Health (MOH) in all the 31 provinces in 2000 [33,34]. About 820 pathology departments report cancer cases to the cancer office manually or electronically every three months. The cancer office evaluates the quality of these data and after removing the duplicates, publishes the annual

Results

727 preliminary questions were developed. In addition, experts suggested 68 new questions. After evaluation, 546 questions were finalized. At the end, four checklists were developed for evaluation of 1) PBCR guidelines, 2) governance and infrastructure, 3) information technology and 4) data analysis and reporting. In addition, four questionnaires were developed for assessment of the knowledge and practices of PBCR staff at several levels including pathology departments, health care centers, and

Discussion

International standards were used and comprehensive tools were developed for external evaluation of the Iranian PBCR. The evaluation package consists of eight checklists and questionnaires.

Many researchers conducted evaluation studies on data quality in cancer registration. Some studies introduced evaluation criteria. For example, the European Structured Reviews of Cancer Registries consider several criteria such as organizational structure, budget, staff, PBCR processes, IT, security and

Conclusion

The comprehensive tools of this study can be used for evaluation of different aspects of PBCRs, including the inputs, processes and outputs of the program. In particular, it can be used to evaluate the guidelines, IT, governance and infrastructure, reports and dissemination, as well as knowledge and practice of the staff of PBCRs. Coverage of the cancer registration is low in most low and middle-income countries, and several initiatives have been started to launch a PBCR in these countries.

Authors’ contributions

A.Sh, K. Z, and A. N designed the study. A.Sh, K. Z, A. N, Z. S, M. H and M. G were responsible for developing preliminary tools and questions. All of the authors were responsible for data gathering and analysis. A. Sh drafted the manuscript. All of the authors revised the manuscript critically. All of the authors read the manuscript and approved it for submission.

Funding

This work was supported by the World Health Organization [Grant no: EM-IRA-2012-APW-087] and Cancer Research Network of I.R of Iran. The funding sources were not involved in the research or preparation of the manuscript.

Conflicts of interest

None.

Summary points

What was already known on this topic:

  • Most methods developed for evaluation of population-based cancer registries focused on the data and outputs of the registries.

  • There are few comprehensive methodologies for evaluation of population-based cancer registries in low- and middle-income countries.

What this study added to our knowledge:

  • A consensus-based, comprehensive evaluation package for population-based cancer registry in Iran was developed; however, it can be tailored for

Acknowledgement

We would like to thank all the national experts who contributed in the validation phase.

References (36)

  • F. Bray et al.

    Planning and Developing Population-Based Cancer Registration in Low- and Middle-Income Settings

    (2015)
  • D.M. Parkin

    The role of cancer registries in cancer control

    Int. J. Clin. Oncol.

    (2008)
  • Y.A. Leal et al.

    The importance of registries in cancer control

    salud pública de méxico

    (2016)
  • David Roder et al.

    Changing roles of population-based cancer registries in Australia

    Aust Health Rev.

    (2015)
  • L.G. Cuervo et al.

    Evaluation of institutional cancer registries in Colombia

    Revista Panamericana de Salud Pública

    (1999)
  • P.B. McGarvey et al.

    Informatics and data quality at collaborative multicenter breast and colon cancer family registries

    J. Am. Med. Inf. Assoc.

    (2012)
  • D.G. Arts et al.

    Defining and improving data quality in medical registries: a literature review, case study, and generic framework

    J. Am. Med. Inf. Assoc.

    (2002)
  • L. Clarkson

    An Evaluation of the Georgia Comperehensive Cancer Registry: Improving an Established System

    (2007)
  • Cited by (10)

    • Regional COVID-19 registry in Khuzestan, Iran: A study protocol and lessons learned from a pilot implementation

      2021, Informatics in Medicine Unlocked
      Citation Excerpt :

      A financial contract was signed with all these individuals to participate in the registry program. We developed several preventive, detective, and corrective actions adopted from literature to ensure data quality [32]. Table 4 shows our measures for these three-controls.

    • Persistence of data-driven knowledge to predict breast cancer survival

      2019, International Journal of Medical Informatics
      Citation Excerpt :

      As new applications of data-driven machine learning methods to predict survival of BC patients appear and new knowledge is generated, it becomes more necessary to discriminate between those findings which are valid over time from those which are only temporarily valid, and in this second case, determine which are their validity times. Population-based cancer databases exist for data analysis [24]. One of the most recent works using the SEER database [6] arrived to fifteen relevant conclusions on the prediction of survival of BC patients (Table 1).

    View all citing articles on Scopus
    View full text