Development and evaluation of an osteoarthritis risk model for integration into primary care health information technology

Highlights

• Osteoarthritis risk can be estimated from primary care electronic medical data.

• Challenges in defining risk indicators and outcomes can be mitigated.

• EMR data support development and evaluation of health informatics systems.

Abstract

Background

We developed and evaluated a prognostic prediction model that estimates osteoarthritis risk for use by patients and practitioners that is designed to be appropriate for integration into primary care health information technology systems. Osteoarthritis, a joint disorder characterized by pain and stiffness, causes significant morbidity among older Canadians. Because our prognostic prediction model for osteoarthritis risk uses data that are readily available in primary care settings, it supports targeting of interventions delivered as part of clinical practice that are aimed at risk reduction.

Methods

We used the CPCSSN (Canadian Primary Sentinel Surveillance Network) database, which contains aggregated electronic health information from a cohort of primary care practices, to develop and evaluate a prognostic prediction model to estimate 5-year osteoarthritis risk, addressing contextual challenges of data availability and missingness. We constructed a retrospective cohort of 383,117 eligible primary care patients who were included in the cohort if they had an encounter with their primary care practitioner between 1 January 2009 and 31 December 2010. Patients were excluded if they had a diagnosis of osteoarthritis prior to their first visit in this time period. Incident cases of osteoarthritis were observed. The model was constructed to predict incident osteoarthritis based on age, sex, BMI, previous leg injury, and osteoporosis. Evaluation of the model used internal 10-fold cross-validation; we argue that internal validation is particularly appropriate for a model that is to be integrated into the same context from which the data were derived.

Results

The resulting prediction model for 5-year risk of osteoarthritis diagnosis demonstrated state-of-the-art discrimination (estimated AUROC 0.84) and good calibration (assessed visually.) The model relies only on information that is readily available in Canadian primary care settings, and hence is appropriate for integration into Canadian primary care health information technology.

Conclusions

If the contextual challenges arising when using primary care electronic medical record data are appropriately addressed, highly discriminative models for osteoarthritis risk may be constructed using only data commonly available in primary care. Because the models are constructed from data in the same setting where the model is to be applied, internal validation provides strong evidence that the resulting model will perform well in its intended application.

Introduction

Prognostic prediction models (PPMs) estimate a patient’s risk of disease development [1,2] based on various predictors [3,4]. Predictors may include patient demographics (such as age and sex), family history, lifestyle factors (such as smoking status or physical activity level), prior medical conditions, laboratory test results, radiographic imaging, or genetic markers [5]. In turn, health care practitioners and patients can make decisions informed by disease risk [6,7]. For example, a patient found to be at high risk of lung cancer may be advised by their health care provider to quit smoking. Several PPMs estimate a patient’s risk of developing osteoarthritis [[8], [9], [10], [11]]; however, all existing PPMs for osteoarthritis that we identified require information on predictors that are not routinely collected in primary care, such as the Kellgren and Lawrence grade (which requires radiographic imaging), and hence are not suitable for integration into primary care health information systems. Using existing data within primary care electronic medical records (EMRs) instead would eliminate the need for collection of additional, oftentimes burdensome, measures and would enable real-time risk estimation at the point of care.

There is the potential for significant benefit to be derived by deploying such a risk engine in primary care. Affecting an estimated 13 % of adults over the age of 20, osteoarthritis causes significant morbidity in Canada [12]. This estimate increases to 29 % in adults 70 years of age and older who receive primary care [13]. Symptoms of osteoarthritis include joint pain and stiffness [14], commonly affecting the joints of the hands, neck, lower back, hips, and knees. Osteoarthritis treatment largely consists of symptom management (e.g., non-steroidal anti-inflammatory medications for pain management), rather than treatment of underlying disease mechanisms [15]. Total joint replacement is often required after significant degradation of the affected joint. To mitigate this burden, prevention strategies have shown potential in reducing the incidence of osteoarthritis. For example, a diet and exercise program aimed at weight loss reduced the incidence of osteoarthritis, though not statistically significantly [16]. Injury prevention programs have been suggested as a potential strategy to prevent osteoarthritis [17]. Interventions such as these may be improved by selectively targeting those at the greatest risk of osteoarthritis in order to reduce their risk. To perform this selective targeting, individualized risk estimates for osteoarthritis are required.

Designing and evaluating a PPM specifically in the primary care context leads to two important design decisions: 1) the PPM should use only data that are easily available in the primary care context where it is to be deployed, for example those that exist in EMRs already, and 2) evaluation of the PPM should reflect the population where it is to be deployed, that is, in primary care encounters. Researchers have begun to recognize the value of EMR data for research purposes more generally [18]; however, data quality within these databases remains uncertain [19]. Issues such as implausible data and missing data are common in EMR data. When working with EMR data, researchers must address these contextual challenges [20].

In this work, we developed and validated a PPM to estimate a patient’s five-year risk of osteoarthritis development using primary care EMR data. Ultimately, we see this model being developed into a purpose-built tool to be used routinely by primary care practitioners during patient encounters to: 1) deliver a quantitative assessment of osteoarthritis risk in patients where the patient and/or primary care practitioner is concerned about osteoarthritis risk; and 2) act as a passive risk screening tool to identify high-risk patients who may have gone undetected otherwise. We are confident that our learnings from this work will be of use to others who are designing and evaluating PPMs using EMR from and for primary care.