Skip to main content
SpringerLink
Log in

Futility of Cluster Designs at Individual Hospitals to Study Surgical Site Infections and Interventions Involving the Installation of Capital Equipment in Operating Rooms

  • Systems-Level Quality Improvement
  • Published:
Journal of Medical Systems Aims and scope Submit manuscript

Abstract

Anesthesia workspaces are integral components in the chains of many intraoperative bacterial transmission events resulting in surgical site infections (SSI). Matched cohort designs can be used to compare SSI rates among operating rooms (ORs) with or without capital equipment purchases (e.g., new anesthesia machines). Patients receiving care in intervention ORs (i.e., with installed capital equipment) are matched with similar patients receiving care in ORs lacking the intervention. We evaluate statistical power of an alternative design for clinical trials in which, instead, SSI incidences are compared directly among ORs (i.e., the ORs form the clusters) at single hospitals (e.g., the 5 ORs with bactericidal lights vs. the 5 other ORs). Data used for parameter estimates were SSI for 24 categories of procedures among 338 hospitals in the State of California, 2015. Estimated statistical power was ≅8.4% for detecting a reduction in the incidence of SSI from 3.6% to 2.4% over 1 year with 5 intervention ORs and 5 control ORs. For ≅80% statistical power, >20 such hospitals would be needed to complete a study in 1 year. Matched paired cluster designs pair similar ORs (e.g., 2 cardiac ORs, 1 to intervention and 1 to control). With 5 pairs, statistical power would be even less than the estimated 8.4%. Cluster designs (i.e., analyses by OR) are not suitable for comparing SSI among ORs at single hospitals. Even though matched cohort designs are non-randomized and thus have lesser validity, matching patients by their risk factors for SSI is more practical.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. These were calculated using α = 0.05 as if the 6000 patients could, in practice, be randomized to ORs [1]. Such a randomization cannot be accomplished for a host of appropriate reasons related to physical room constraints, surgeons having lists of consecutive cases to perform, urgent surgical cases needing to be completed, etc. [1]. For the STATA function, see Reference [1] Supplemental content Section #4 (e.g., at https://FDshort.com/SSIpublic).

  2. Use of 5 ORs and a historical period with no intervention, analyzed by segmented regression, would achieve statistical power less than for ORs each with 600 patients per OR. The latter has estimated statistical power of 8.9%, only 0.5% greater than without historical data.

  3. Functionally this would require patients to be randomized to ORs, ORs to be used interchangeably at all times, and there to be no infection transmission between successive patients in ORs. None is true [1].

  4. For this calculation, the numerator of the effect size would be the difference, 0.036–0.024 = 0.012. The standard error ≅ (0.036 × (1–0.036) / 600 + 0.024 × (1–0.024) / 600)0.5 = 0.0098. The ratio of 0.012 / 0.0098 = 1.22.

References

  1. Dexter F, Ledolter J, Epstein RH, Loftus RW. Importance of operating room case scheduling on analyses of observed reductions in surgical site infections from the purchase and installation of capital equipment in operating rooms. Am J Infect Control. 2020.

  2. Schmidt E, Dexter F, Hermann J, Godding JD, Hadder B, Loftus RW. Assessment of anesthesia machine redesign on cleaning of the anesthesia machine using surface disinfection wipes. Am J Infect Control. 2020.

  3. Dräger Perseus A500. Available from: https://www.draeger.com/Products/Content/perseus-br-9067209-en-master.pdf#page=3. Accessed September 2, 2018.

  4. Al-Waked, R., Effect of ventilation strategies on infection control inside operating theatres. Eng Appl Comp Fluid. 4:1–16, 2010.

    Google Scholar 

  5. Thomas, A. M., and Simmons, M. J., The effectiveness of ultra-clean air operating theatres in the prevention of deep infection in joint arthroplasty surgery. Bone Joint J. 100-B(10):1264–1269, 2018.

    Article  CAS  Google Scholar 

  6. Indigo-Clean visible light disinfection. Available from: http://indigo-clean.com/how-it-works. Accessed September 2, 2018.

  7. Murrell, L. J., Hamilton, E. K., Johnson, H. B., and Spencer, M., Influence of a visible-light continuous environmental disinfection system on microbial contamination and surgical site infections in an orthopedic operating room. Am. J. Infect. Control 47:804–810, 2019.

    Article  Google Scholar 

  8. Weiser, M. C., Shemesh, S., Chen, D. D., Bronson, M. J., and Moucha, C. S., The effect of door opening on positive pressure and airflow in operating rooms. J Am Acad Orthop Surg. 26:e105–e113, 2018.

    Article  Google Scholar 

  9. Epstein, R. H., and Dexter, F., Database quality and access issues relevant to research using anesthesia information management system data. Anesth. Analg. 127:105–114, 2018.

    Article  Google Scholar 

  10. Cook, J. A., Bruckner, T., MacLennan, G. S., and Seiler, C. M., Clustering in surgical trials--database of intracluster correlations. Trials. 13:2, 2012.

    Article  Google Scholar 

  11. California Department of Public Health, Surgical Site Infections (SSIs) for Operative Procedures in Healthcare, SSIs for 24 operative procedures, 2015. Available from: https://data.chhs.ca.gov/dataset/surgical-site-infections-ssis-for-5-operative-procedures-in-hospitals. Downloaded August 19, 2018.

  12. Klar, N., and Donner, A., The merits of matching in community intervention trials: a cautionary tale. Stat. Med. 16:1753–1764, 1997.

    Article  CAS  Google Scholar 

  13. Loftus, R. W., Brown, J. R., Koff, M. D., Reddy, S., Heard, S. O., Patel, H. M., Fernandez, P. G., Beach, M. L., Corwin, H. L., Jensen, J. T., Kispert, D., Huysman, B., Dodds, T. M., Ruoff, K. L., and Yeager, M. P., Multiple reservoirs contribute to intraoperative bacterial transmission. Anesth. Analg. 114:1236–1248, 2012.

    Article  Google Scholar 

  14. Dexter, F., Epstein, R. H., and Penning, D. H., Late first-case of the day starts do not cause greater minutes of over-utilized time at an endoscopy suite with 8-hour workdays and late running rooms. A historical cohort study. J. Clin. Anesth. 59:18–25, 2020.

    Article  Google Scholar 

  15. Solovy, A., Benchmarking guide 99. Hosp. Health Netw. 73:49–62, 1999.

    CAS  PubMed  Google Scholar 

  16. Dexter, F., Jarvie, C., and Epstein, R. H., At most hospitals in the State of Iowa, most surgeons’ daily lists of elective cases include only 1 or 2 cases: individual surgeons’ percentage operating room utilization is a consistently unreliable metric. J. Clin. Anesth. 42:88–92, 2017.

    Article  Google Scholar 

  17. Ribeiroa, D. C., Milosavljevicb, S., and Abbottc, J. H., Sample size estimation for cluster randomized controlled trials. Musculoskelet Sci Pract. 34:108–111, 2018.

    Article  Google Scholar 

  18. Tian, L., Inferences on the common coefficient of variation. Stat. Med. 24:2213–2220, 2005.

    Article  Google Scholar 

  19. Koff, M. D., Brown, J. R., Marshall, E. J., O'Malley, A. J., Jensen, J. T., Heard, S. O., Longtine, K., O'Neill, M., Longtine, J., Houston, D., Robison, C., Moulton, E., Patel, H. M., and Loftus, R. W., Frequency of hand decontamination of intraoperative providers and reduction of postoperative healthcare-associated infections: a randomized clinical trial of a novel hand hygiene system. Infect. Control Hosp. Epidemiol. 37:888–895, 2016.

    Article  Google Scholar 

  20. STATA Power analysis for a two-sample proportions test. Available from: https://www.stata.com/manuals13/psspowertwoproportions.pdf. .

  21. Hemming, K., Girling, A. J., Sitch, A. J., Marsh, J., and Lilford, R. J., Sample size calculations for cluster randomised controlled trials with a fixed number of clusters. BMC Med. Res. Methodol. 11:102, 2011.

    Article  Google Scholar 

  22. Ledolter, J., Economic field experiments: comments on design efficiency, sample size and statistical power. J Econ Manag. 9:271–290, 2013.

    Google Scholar 

  23. Eldridge, S. M., Ukoumunne, O. C., and Carlin, J. B., The intra-cluster correlation coefficient in cluster randomized trials: a review of definitions. Int. Stat. Rev. 77:378–394, 2009.

    Article  Google Scholar 

  24. National Healthcare Safety Network, Surveillance for Surgical Site Infection (SSI) Events, 2018 Operative Procedure Code Documents. Available from: https://www.cdc.gov/nhsn/acute-care-hospital/ssi/index.html. Accessed August 19, 2018.

  25. Wu, S., Crespi, C. M., and Wong, W. K., Comparison of methods for estimating the intraclass correlation coefficient for binary responses in cancer prevention cluster randomized trials. Contemp Clin Trials. 33:869–880, 2012.

    Article  Google Scholar 

  26. Birgand, G., Lepelletier, D., Baron, G., Barrett, S., Breier, A. C., Buke, C., Markovic-Denic, L., Gastmeier, P., Kluytmans, J., Lyytikainen, O., Sheridan, E., Szilagyi, E., Tacconelli, E., Troillet, N., Ravaud, P., and Lucet, J. C., Agreement among healthcare professionals in ten European countries in diagnosing case-vignettes of surgical-site infections. PLoS One 8(7):e68618, 2013:9.

    Article  CAS  Google Scholar 

  27. Separation. https://en.wikipedia.org/wiki/Separation_(statistics). Accessed August 17, 2019.

  28. Kleinman K. Firth logistic regression. https://www.r-bloggers.com/example-8-15-firth-logistic-regression/. Accessed August 17, 2019.

  29. STATA Power analysis for a two-sample proportions test, cluster randomized design. Available from: https://www.stata.com/manuals/pss-2powertwoproportionscluster.pdf. .

  30. Petersen-Fitts, G., Gambone, A., Sherwood, A., Whaley, J., Katz, D. L., Lima, D. J. L., and Sabesan, V., Comparison of microbial count on various surfaces in operating rooms at different times of the day. J. Surg. Orthop. Adv. 27:281–285, 2018.

    PubMed  Google Scholar 

  31. Balzer, L. B., Petersen, M. L., van der Laan, M. J., and SEARCH Consortium, Adaptive pair-matching in randomized trials with unbiased and efficient effect estimation. Stat. Med. 34:999–1011, 2015.

    Article  Google Scholar 

  32. Masursky, D., Dexter, F., and Nussmeier, N. A., Operating room nursing directors’ influence on anesthesia group operating room productivity. Anesth. Analg. 107:1989–1996, 2008.

    Article  Google Scholar 

  33. Austin, P. C., Grootendorst, P., and Anderson, G. M., A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study. Stat. Med. 26:734–753, 2007.

    Article  Google Scholar 

  34. Dexter F, Epstein RH, Gostine AL, Penning DH, Loftus RW. Benefit of systematic selection of pairs of cases matched by surgical specialty for surveillance of bacterial transmission in operating rooms. Am J Infect Control. 2020.

  35. Dexter, F., Epstein, R. H., Bayman, E. O., and Ledolter, J., Estimating surgical case durations and making comparisons among facilities: identifying facilities with lower anesthesia professional fees. Anesth. Analg. 116:1103–1115, 2013.

    Article  Google Scholar 

  36. Dexter, F., Dexter, E. U., and Ledolter, J., Importance of appropriately modeling procedure and duration in logistic regression studies of perioperative morbidity and mortality. Anesth. Analg. 13:1197–1201, 2011.

    Article  Google Scholar 

  37. Dexter, F., Wachtel, R. E., and Epstein, R. H., Decreasing the hours that anesthesiologist and nurse anesthetists work late by making decisions to reduce the hours of over-utilized operating room time. Anesth. Analg. 122:831–842, 2016.

    Article  CAS  Google Scholar 

  38. Dexter, F., Epstein, R. D., Traub, R. D., and Xiao, Y., Making management decisions on the day of surgery based on operating room efficiency and patient waiting times. Anesthesiology. 101:1444–1453, 2004.

    Article  Google Scholar 

  39. O’Sullivan, C. T., and Dexter, F., Assigning surgical cases with regional anesthetic blocks to anesthetists and operating rooms based on operating room efficiency. AANA J. 74:213–218, 2006.

    PubMed  Google Scholar 

  40. Dexter, F., Willemsen-Dunlap, A., and Lee, J. D., Operating room managerial decision-making on the day of surgery with and without computer recommendations and status displays. Anesth. Analg. 105:419–429, 2007.

    Article  Google Scholar 

  41. Dexter, F., Wachtel, R. E., and Epstein, R. H., Event-based knowledge elicitation of operating room management decision-making using scenarios adapted from information systems data. BMC Med Inform Decis Mak. 11:2, 2011.

    Article  Google Scholar 

  42. Dexter, F., and Traub, R. D., How to schedule elective surgical cases into specific operating rooms to maximize the efficiency of use of operating room time. Anesth. Analg. 94:933–942, 2002.

    Article  Google Scholar 

  43. Shi, P., Dexter, F., and Epstein, R. H., Comparing policies for case scheduling within one day of surgery by Markov chain models. Anesth. Analg. 122:526–538, 2016.

    Article  Google Scholar 

  44. Van Houdenhoven, M., van Oostrum, J. M., Hans, E. W., Wullink, G., and Kazemier, G., Improving operating room efficiency by applying bin-packing and portfolio techniques to surgical case scheduling. Anesth. Analg. 105:707–714, 2007.

    Article  Google Scholar 

  45. McIntosh, C., Dexter, F., and Epstein, R. H., The impact of service-specific staffing, case scheduling, turnovers, and first-case starts on anesthesia group and operating room productivity: tutorial using data from an Australian hospital. Anesth. Analg. 103:1499–1516, 2006.

    Article  Google Scholar 

  46. Dexter, F., Traub, R. D., and Macario, A., How to release allocated operating room time to increase efficiency: predicting which surgical service will have the most underutilized operating room time. Anesth. Analg. 96:507–512, 2003.

    PubMed  Google Scholar 

  47. Tanaka, M., Lee, J., Ikai, H., and Imanaka, Y., Development of efficiency indicators of operating room management for multi-institutional comparisons. J. Eval. Clin. Pract. 19:335–341, 2013.

    Article  Google Scholar 

  48. Moazzez, A., and de Virgilio, C., Role of surgical services in profitability of hospitals in California: an analysis of office of statewide health planning and development annual financial data. Am. Surg. 82:894–897, 2016.

    PubMed  Google Scholar 

  49. HAI data. https://www.cdc.gov/hai/data/index.html. Accessed August 17, 2019.

  50. Government of Western Australia, Department of Health, Healthcare associated surgical site infections. https://ww2.health.wa.gov.au/Reports-and-publications/Healthcare-associated-infections/data?report=ssi_table. Accessed February 25, 2020.

  51. Annual epidemiological report for 2016, surgical site infections. https://ecdc.europa.eu/sites/portal/files/documents/AER_for_2016-SSI.pdf#page=3. Accessed June 18, 2019.

  52. Sakamoto, F., Asano, K., Sakihama, T., Saint, S., Greene, M. T., Patel, P., Ratz, D., and Tokuda, Y., Changes in health care-associated infection prevention practices in Japan: Results from 2 national surveys. Am. J. Infect. Control 47:65–68, 2019.

    Article  Google Scholar 

Download references

Funding

Funding for this project was provided solely from departmental sources.

Author information

Authors and Affiliations

  1. Department of Anesthesia, University of Iowa, 200 Hawkins Drive, 6 JCP, Iowa City, IA, 52242, USA

    Franklin Dexter & Randy W. Loftus

  2. Department of Business Analytics, University of Iowa, Iowa City, IA, USA

    Johannes Ledolter

  3. Departments of Anesthesiology and Perioperative Medicine & Pain Management, University of Miami, Miami, FL, USA

    Richard H. Epstein

Authors
  1. Franklin Dexter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Johannes Ledolter
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Richard H. Epstein
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Randy W. Loftus
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Franklin Dexter.

Ethics declarations

Conflict of Interest

Dräger, Kenall Manufacturing, and the Anesthesia Patient Safety Foundation have funded studies being performed at the University of Iowa with statistical issues considered in this paper. Dr. Loftus has presented at educational meetings sponsored by Kenall Manufacturing.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on Systems-Level Quality Improvement

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dexter, F., Ledolter, J., Epstein, R.H. et al. Futility of Cluster Designs at Individual Hospitals to Study Surgical Site Infections and Interventions Involving the Installation of Capital Equipment in Operating Rooms. J Med Syst 44, 82 (2020). https://doi.org/10.1007/s10916-020-01555-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-020-01555-0

Keywords

Search

Navigation