Abstract
Interrupted time series (ITS) design is a quasi-experimental approach for evaluating the impact of an intervention in public health research using segmented linear regression (SLR) models. Usually, aggregated data across multiple time points before and after an intervention are compared to detect a change in intercept, slope, or both; however, the use of aggregated data can lead to an ecological fallacy, imprecise estimates, loss of power, and spurious inferences. We formulated three models with/without measurement error in the dependent variable to address different data limitations resulted from aggregation. Adopting the Bayesian hierarchical methodology for three standard SLR models from an ITS design, we compared performances of the varying pre-intervention intercept model (VPIM), varying intercept model (VIM) and measurement error model (MEM) with a non-hierarchical model (NHM) using real-life data and simulation studies. The MEM first estimates true value using observed data and standard deviation, then regresses the independent variables on the estimated true values. The results demonstrated the suitability of the hierarchical models through sustained improvement in model performance and parameter estimates over the NHM. The VPIM and VIM provided precise estimates modeling the population of clusters and pooling information across parameters. The measurement error assumption, along with the Bayesian model’s hierarchical formulation and generative nature, helped stabilize the unstable values of the dependent variable based on observed data and standard deviations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and materials
Code availability
The R codes are available upon request from the first author.
References
Barone-Adesi, F., Gasparrini, A., Vizzini, L., Merletti, F., Richiardi, L.: Effects of Italian smoking regulation on rates of hospital admission for acute coronary events: a country-wide study. PLoS ONE 6(3), e17419 (2011). https://doi.org/10.1371/journal.pone.0017419
Boel, J., et al.: Impact of antibiotic restriction on resistance levels of Escherichia coli: a controlled interrupted time series study of a hospital-wide antibiotic stewardship programme. J. Antimicrob. Chemother. 71(7), 2047–2051 (2016). https://doi.org/10.1093/jac/dkw055
Box, G., Jenkins, G., Reinsel, G., Ljung, G.: Time Series Analysis: Forecasting and Control, 5th edn. Wiley, New Jersey (2015)
Bürkner, P.-C.: Advanced Bayesian Multilevel Modeling with the R Package brms. R J. 10(1), 395 (2018). https://doi.org/10.32614/RJ-2018-017. arXiv:1705.11123
Bürkner, P.-C.: brms : an R package for bayesian multilevel models using stan. J. Stat. Softw. (2017). https://doi.org/10.18637/jss.v080.i01
Carpenter, B., et al.: Stan: a probabilistic programming language. J. Stat. Softw. (2017). https://doi.org/10.18637/jss.v076.i01
Clark, J.S.: Uncertainty and variability in demography and population growth: a hierarchical approach. Spec. Feature Ecol. 84(6), 1370–1381 (2003). https://doi.org/10.1890/0012-9658
Clayton, D., Kaldor, J.: Empirical bayes estimates of age-standardized relative risks for use in disease mapping. Biometrics 43(3), 671 (1987). https://doi.org/10.2307/2532003
Cruz, M., Bender, M., Ombao, H.: A robust interrupted time series model for analyzing complex health care intervention data. Stat. Med. 36(29), 4660–4676 (2017). https://doi.org/10.1002/sim.7443. (arXiv:1707.01861)
Dayer, M.J., et al.: Incidence of infective endocarditis in England, 2000–13: a secular trend, interrupted time-series analysis. Lancet 385(9974), 1219–1228 (2015). https://doi.org/10.1016/S0140-6736(14)62007-9
Donohue, J.J., Aneja, A., Weber, K.D.: Right-to-carry laws and violent crime: a comprehensive assessment using panel data and a state-level synthetic control analysis. J. Empiri. Legal Stud. 16(2), 198–247 (2019). https://doi.org/10.1111/jels.12219
Dowell, D., et al.: Changes in fluoroquinolone use for gonorrhea following publication of revised treatment guidelines. Am. J. Public Health 102(1), 148–155 (2012). https://doi.org/10.2105/AJPH.2011.300283
Etchepare, F., Pambrun, E., Verdoux, H., Tournier, M.: Trends in patterns of antidepressant use in older general population between 2006 and 2012 following publication of practice guidelines. Int. J. Geriatr. Psychiatry 32(8), 849–859 (2017). https://doi.org/10.1002/gps.4536
Ewusie, J.E., et al.: Methods, applications and challenges in the analysis of interrupted time series data: a scoping review. J. Multidiscip. Healthc. 13, 411–423 (2020). https://doi.org/10.2147/JMDH.S241085
Gabry, J., Simpson, D., Vehtari, A., Betancourt, M., Gelman, A.: Visualization in Bayesian workflow. J. R. Stat. Soc. Ser. A (Stat. Soc.) 182(2), 389–402 (2019). https://doi.org/10.1111/rssa.12378. (arXiv:1709.01449)
Garrett, T.A.: Aggregated versus disaggregated data in regression analysis: implications for inference. Econ. Lett. 81(1), 61–65 (2003). https://doi.org/10.1016/S0165-1765(03)00149-6
Gebski, V., Ellingson, K., Edwards, J., Jernigan, J., Kleinbaum, D.: Modelling interrupted time series to evaluate prevention and control of infection in healthcare. Epidemiol. Infect. 140(12), 2131–2141 (2012). https://doi.org/10.1017/S0950268812000179
Gelman, A.: The connection between varying treatment effects and the crisis of unreplicable research. J. Manage. 41(2), 632–643 (2015). https://doi.org/10.1177/0149206314525208
Hox, J.J.: Multilevel analysis: techniques and applications, 2nd edn. Lawrence Erlbaum, Mahwah (2010)
Jandoc, R., Burden, A.M., Mamdani, M., Lévesque, L.E., Cadarette, S.M.: Interrupted time series analysis in drug utilization research is increasing: systematic review and recommendations. J. Clin. Epidemiol. 68(8), 950–956 (2015). https://doi.org/10.1016/j.jclinepi.2014.12.018
Jiao, Y., Hayes, C., Cortés, E.: Hierarchical Bayesian approach for population dynamics modelling of fish complexes without species-specific data. ICES J. Mar. Sci. 66(2), 367–377 (2009). https://doi.org/10.1093/icesjms/fsn162
Kastner, M., et al.: A knowledge translation tool improved osteoporosis disease management in primary care: an interrupted time series analysis. Implement. Sci. 9(1), 109 (2014). https://doi.org/10.1186/s13012-014-0109-9
Kleiber, C., Zeileis, A.: Applied econometrics with R, Springer New York, New York (2008). http://link.springer.com/10.1007/978-0-387-77318-6
Liu, B., et al.: Outcomes of mobilisation of vulnerable elders in Ontario (MOVE ON): a multisite interrupted time series evaluation of an implementation intervention to increase patient mobilisation. Age Ageing 47(1), 112–119 (2018). https://doi.org/10.1093/ageing/afx128
Lopez Bernal, J., Cummins, S., Gasparrini, A.: Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int. J. Epidemiol. (2016). https://doi.org/10.1093/ije/dyw098
Lott, J.R., Mustard, D.: in Crime, deterrence, and right-to-carry concealed handguns, pp. 129–200 (2008). http://www.crcnetbase.com/doi/abs/10.4324/9780203930885.ch8
Lyman, G.H., Kuderer, N.M.: The strengths and limitations of meta-analyses based on aggregate data. BMC Med. Res. Methodol. 5(1), 14 (2005). https://doi.org/10.1186/1471-2288-5-14
MacNab, Y.C.: On identification in Bayesian disease mapping and ecological-spatial regression models. Stat. Methods Med. Res. 23(2), 134–155 (2014). https://doi.org/10.1177/0962280212447152
Makowski, D., Ben-Shachar, M., Lüdecke, D.: bayestestR: describing effects and their uncertainty, existence and significance within the Bayesian framework. J. Open Sour. Softw. 4(40), 1541 (2019). https://doi.org/10.21105/joss.01541
McElreath, R.: Statistical Rethinking. Chapman and Hall/CRC, Boca Raton (2018) https://www.taylorfrancis.com/books/9781315362618
Morris, T.P., White, I.R., Crowther, M.J.: Using simulation studies to evaluate statistical methods. Stat. Med. 38, 2074–2102 (2019). https://doi.org/10.1002/sim.8086
Penfold, R.B., Zhang, F.: Use of interrupted time series analysis in evaluating health care quality improvements. Acad. Pediatr. 13(6), S38–S44 (2013). https://doi.org/10.1016/j.acap.2013.08.002
Piray, P., Dezfouli, A., Heskes, T., Frank, M.J., Daw, N.D.: Hierarchical Bayesian inference for concurrent model fitting and comparison for group studies. PLOS Comput. Biol. 15(6), e1007043 (2019). https://doi.org/10.1371/journal.pcbi.1007043
R Core Team. R: A Language and Environment for Statistical Computing (2021). https://www.r-project.org/
Ramsay, C.R., Matowe, L., Grilli, R., Grimshaw, J.M., Thomas, R.E.: Interrupted time series designs in health technology assessment: lessons from two systematic reviews of behavior change strategies. Int. J. Technol. Assess. Health Care 19(4), 613–623 (2003). https://doi.org/10.1017/S0266462303000576
Rana, M. et al.: In: Chaubey, Y.P., Lahmiri, S., Nebebe, F., Sen, A. (eds.) Modeling obesity rate with spatial auto-correlation: a case study. Applied Statistics and Data Science, vol. 375, Ch. 3, pp. 53–77. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86133-9_3
Robinson, W.: Ecological correlations and the behavior of individuals. Int. J. Epidemiol. 38(2), 337–341 (2009). https://doi.org/10.1093/ije/dyn357
Stan Development Team. RStan: the R interface to Stan (2020). http://mc-stan.org/
Wagner, A.K., Soumerai, S.B., Zhang, F., Ross-Degnan, D.: Segmented regression analysis of interrupted time series studies in medication use research. J. Clin. Pharm. Ther. 27(4), 299–309 (2002). https://doi.org/10.1046/j.1365-2710.2002.00430.x
Wang, J.J.J., Walter, S., Grzebieta, R., Olivier, J.: A comparison of statistical methods in interrupted time series analysis to estimate an intervention effect. Proceedings of the 2013 Australasian Road Safety Research, Policing & Education Conference (2013)
Watanabe, S. Asymptotic equivalence of bayes cross validation and widely applicable information criterion in singular learning theory. J. Mach. Learn. Res. (2010). arXiv:1004.2316
Wickham, H. ggplot2: Elegant Graphics for Data Analysis Use R! (Springer International Publishing, Cham, 2016). http://link.springer.com/10.1007/978-3-319-24277-4
Wickham, H., et al.: Welcome to the Tidyverse. J. Open Sour. Softw. 4(43), 1686 (2019). https://doi.org/10.21105/joss.01686
Zhang, F., Wagner, A.K., Soumerai, S.B., Ross-Degnan, D.: Methods for estimating confidence intervals in interrupted time series analyses of health interventions. J. Clin. Epidemiol. 62(2), 143–148 (2009). https://doi.org/10.1016/j.jclinepi.2008.08.007
Acknowledgements
The authors would like to thank Shahedul A Khan, University of Saskatchewan for helpful feedback on the first manuscript draft.
Funding
Not applicable.
Author information
Authors and Affiliations
Contributions
MR conceptualized the study and performed the statistical analysis and simulation. JK and SP were the team lead for the Antimicrobial Stewardship Program at Saskatoon and shared the data for the first example. MR wrote the first manuscript draft. Both SP and JK contributed to manuscript preparation. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors have declared no conflict of interest.
Ethical approval
Not applicable.
Consent to participate
Not applicable.
Consent for publication
The authors confirm that there are no similar articles published or submitted for publication elsewhere.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Material
Supplementary Material
Impact of measurement error assumption on robbery rate from More Guns, Less Crime data. The dashed line and solid line present estimate from varying effects model and measurement error model, respectively for each state. The shaded area shows \(89\%\) credible interval. (Color figure online)
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Rana, M., Kosar, J. & Peermohamed, S. Bayesian hierarchical models incorporating measurement error for interrupted time series design. Stat Comput 33, 127 (2023). https://doi.org/10.1007/s11222-023-10295-3
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11222-023-10295-3