Skip to main content
SpringerLink
Log in

Importance of Statistical Evidence in Estimating Valid DEA Scores

  • Systems-Level Quality Improvement
  • Published:
Journal of Medical Systems Aims and scope Submit manuscript

Abstract

Data Envelopment Analysis (DEA) allows healthcare scholars to measure productivity in a holistic manner. It combines a production unit’s multiple outputs and multiple inputs into a single measure of its overall performance relative to other units in the sample being analyzed. It accomplishes this task by aggregating a unit’s weighted outputs and dividing the output sum by the unit’s aggregated weighted inputs, choosing output and input weights that maximize its output/input ratio when the same weights are applied to other units in the sample. Conventional DEA assumes that inputs and outputs are used in different proportions by the units in the sample. So, for the sample as a whole, inputs have been substituted for each other and outputs have been transformed into each other. Variables are assigned different weights based on their marginal rates of substitution and marginal rates of transformation. If in truth inputs have not been substituted nor outputs transformed, then there will be no marginal rates and therefore no valid basis for differential weights. This paper explains how to statistically test for the presence of substitutions among inputs and transformations among outputs. Then, it applies these tests to the input and output data from three healthcare DEA articles, in order to identify the effects on DEA scores when input substitutions and output transformations are absent in the sample data. It finds that DEA scores are badly biased when substitution and transformation are absent and conventional DEA models are used.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. A production technology can involve substituted inputs and transformed outputs for which there can be no valid convex combinations of the inputs or outputs of efficient production units. Models prohibiting convex combinations such as Free Disposal Hull (FDH) still implicitly assume the presence of substitution and transformation. In Tulkens [3], for example, his Fig. 2 illustrates an FDH frontier on which the production units have different proportions of the two inputs (so substitutions have occurred), and his Fig. 3 illustrates an FDH frontier on which the production units have differing proportions of the two outputs (so transformations have occurred). Radial DEA models assume that both substitution/transformation and convex combinations are present, as also is shown in the two Tulkens figures. Of course, if there has been no substitution or transformation then the convex combination assumption is irrelevant.

  2. Marginal Rate of Substitution–MRS–is the absolute value of the ratio of one input’s decline to another input’s increase, holding all outputs and all remaining inputs constant. For example, if each 1 unit of decrease in Input A requires an increase of 5 units of Input B in order to maintain a constant output, then the MRS is 1/5.

  3. Marginal Rate of Transformation –MRT–is the absolute value of the ratio of one output’s decline to another output’s increase, holding all inputs and all other outputs constant. For example, if increasing production of one unit of Output A results in a decrease of four units of Output B, then the marginal rate of transformation is 1/4.

References

  1. Banker, R. D., Estimating the most productive scale size using data envelopment analysis. Eur. J. Oper. Res. 17(1):35–44, 1984.

    Article  Google Scholar 

  2. Charnes, A., Cooper, W. W., Golany, B., Seiford, L., and Stutz, J., Foundations of data envelopment analysis for Pareto-Koopmans efficient empirical production functions. J. Econ. 30(1–2):91–107, 1985. doi:10.1016/0304-4076(85)90133-2.

    Article  Google Scholar 

  3. Tulkens, H., On FDH efficiency analysis. J. Prod. Anal. 4(1–2):183–210, 1993. doi:10.1007/BF01073473.

  4. Førsund, F. R., Weight restrictions in DEA: Misplaced emphasis? J. Prod. Anal. 40(3):271–283, 2013. doi:10.1007/s11123-012-0296-9.

    Article  Google Scholar 

  5. Barnum, D., Walton, S., Shields, K., and Schumock, G., Measuring hospital efficiency with data envelopment analysis: Nonsubstitutable vs. substitutable inputs and outputs. J. Med. Syst. 35(6):1393–1401, 2011. doi:10.1007/s10916-009-9416-0.

    Article  PubMed  Google Scholar 

  6. Emrouznejad, A., and Dey, P., Performance measurement in the health sector: Uses of frontier efficiency methodologies and multi-criteria decision making. J. Med. Syst. 35(5):977–979, 2011. doi:10.1007/s10916-010-9622-9.

    Article  PubMed  Google Scholar 

  7. Ramírez-Valdivia, M., Maturana, S., and Salvo-Garrido, S., A multiple stage approach for performance improvement of primary healthcare practice. J. Med. Syst. 35(5):1015–1028, 2011. doi:10.1007/s10916-010-9438-7.

    Article  PubMed  Google Scholar 

  8. Osman, I., Berbary, L., Sidani, Y., Al-Ayoubi, B., and Emrouznejad, A., Data envelopment analysis model for the appraisal and relative performance evaluation of nurses at an intensive care unit. J. Med. Syst. 35(5):1039–1062, 2011. doi:10.1007/s10916-010-9570-4.

    Article  PubMed  Google Scholar 

  9. Kontodimopoulos, N., Papathanasiou, N., Flokou, A., Tountas, Y., and Niakas, D., The impact of non-discretionary factors on DEA and SFA technical efficiency differences. J. Med. Syst. 35(5):981–989, 2011. doi:10.1007/s10916-010-9521-0.

    Article  PubMed  Google Scholar 

  10. Cordero-Ferrera, J., Crespo-Cebada, E., and Murillo-Zamorano, L., Measuring technical efficiency in primary health care: The effect of exogenous variables on results. J. Med. Syst. 35(4):545–554, 2011. doi:10.1007/s10916-009-9390-6.

    Article  PubMed  Google Scholar 

  11. Ketabi, S., Efficiency measurement of cardiac care units of Isfahan hospitals in Iran. J. Med. Syst. 35(2):143–150, 2011. doi:10.1007/s10916-009-9351-0.

    Article  PubMed  Google Scholar 

  12. Barnum, D., Shields, K., Walton, S., and Schumock, G., Improving the efficiency of distributive and clinical services in hospital pharmacy. J. Med. Syst. 35(1):59–70, 2011. doi:10.1007/s10916-009-9341-2.

    Article  PubMed  Google Scholar 

  13. Flokou, A., Kontodimopoulos, N., and Niakas, D., Employing post-DEA cross-evaluation and cluster analysis in a sample of Greek NHS hospitals. J. Med. Syst. 35(5):1001–1014, 2011. doi:10.1007/s10916-010-9533-9.

    Article  PubMed  Google Scholar 

  14. Chuang, C.-L., Chang, P.-C., and Lin, R.-H., An efficiency data envelopment analysis model reinforced by classification and regression tree for hospital performance evaluation. J. Med. Syst. 35(5):1075–1083, 2011. doi:10.1007/s10916-010-9598-5.

    Article  PubMed  Google Scholar 

  15. Rouse, P., Harrison, J., and Turner, N., Cost and performance: Complements for improvement. J. Med. Syst. 35(5):1063–1074, 2011. doi:10.1007/s10916-010-9520-1.

    Article  PubMed  Google Scholar 

  16. Lewis, H., Sexton, T., and Dolan, M., An efficiency-based multicriteria strategic planning model for ambulatory surgery centers. J. Med. Syst. 35(5):1029–1037, 2011. doi:10.1007/s10916-010-9522-z.

    Article  PubMed  Google Scholar 

  17. Blank, J. T., and van Hulst, B., Governance and performance: The performance of Dutch hospitals explained by governance characteristics. J. Med. Syst. 35(5):991–999, 2011. doi:10.1007/s10916-010-9437-8.

    Article  PubMed Central  PubMed  Google Scholar 

  18. Ortiz, J., Meemon, N., Tang, C.-Y., Wan, T. H., and Paek, S., Rural health clinic efficiency and effectiveness: Insight from a nationwide survey. J. Med. Syst. 35(4):671–681, 2011. doi:10.1007/s10916-009-9404-4.

    Article  PubMed  Google Scholar 

  19. Jahangoshai Rezaee, M., Moini, A., and Haji-Ali Asgari, F., Unified performance evaluation of health centers with integrated model of data envelopment analysis and bargaining game. J. Med. Syst. 36(6):3805–3815, 2012. doi:10.1007/s10916-012-9853-z.

    Article  PubMed  Google Scholar 

  20. Chang, D., Chung, J., Sun, K., and Yang, F., A novel approach for evaluating the risk of health care failure modes. J. Med. Syst. 36(6):3967–3974, 2012. doi:10.1007/s10916-012-9868-5.

    Article  PubMed  Google Scholar 

  21. Hadji, B., Meyer, R., Melikeche, S., Escalon, S., and Degoulet, P., Assessing the relationships between hospital resources and activities: A systematic review. J. Med. Syst. 38(10):1–21, 2014. doi:10.1007/s10916-014-0127-9.

    Article  Google Scholar 

  22. Pelone, F., Kringos, D., Romaniello, A., Archibugi, M., Salsiri, C., and Ricciardi, W., Primary care efficiency measurement using data envelopment analysis: A systematic review. J. Med. Syst. 39(1):1–14, 2014. doi:10.1007/s10916-014-0156-4.

    Google Scholar 

  23. Deidda, M., Lupiáñez-Villanueva, F., Codagnone, C., and Maghiros, I., Using data envelopment analysis to analyse the efficiency of primary care units. J. Med. Syst. 38(10):1–10, 2014. doi:10.1007/s10916-014-0122-1.

    Article  Google Scholar 

  24. Girginer, N., Köse, T., and Uçkun, N., Efficiency analysis of surgical services by combined use of data envelopment analysis and gray relational analysis. J. Med. Syst. 39(5):1–9, 2015. doi:10.1007/s10916-015-0238-y.

    Article  Google Scholar 

  25. Rezaee, M. J., and Karimdadi, A., Do geographical locations affect in hospitals performance? A multi-group data envelopment analysis. J. Med. Syst. 2015. doi:10.1007/s10916-015-0278-3.

    PubMed  Google Scholar 

  26. National_Chicken_Council, Wings 2 rule Superbowl XLIX Meat & Poultry, 2015.

  27. Barnum, D., and Gleason, J., Measuring efficiency under fixed proportion technologies. J. Prod. Anal. 35(3):243–262, 2011. doi:10.1007/s11123-010-0194-y.

    Article  Google Scholar 

  28. Charnes, A., Cooper, W. W., and Rhodes, E., Measuring the efficiency of decision making units. Eur. J. Oper. Res. 2(6):429–444, 1978.

    Article  Google Scholar 

  29. Dyson, R. G., Allen, R., Camanho, A. S., Podinovski, V. V., Sarrico, C. S., and Shale, E. A., Pitfalls and protocols in DEA. Eur. J. Oper. Res. 132(2):245–259, 2001. doi:10.1016/S0377-2217(00)00149-1.

    Article  Google Scholar 

  30. Cook, W. D., Tone, K., and Zhu, J., Data envelopment analysis: Prior to choosing a model. Omega 44:1–4, 2014. doi:10.1016/j.omega.2013.09.004.

    Article  Google Scholar 

  31. Coelli, T. J., Rao, D. S. P., O’Donnell, C. J., and Battese, G. E., An introduction to efficiency and productivity analysis, 2nd edition. Springer, New York, 2005.

    Google Scholar 

Download references

Acknowledgments

We gratefully acknowledge financial support for summer research provided by the Dean of the College of Business Administration, University of Illinois at Chicago.

Author information

Authors and Affiliations

  1. College of Business Administration, College of Pharmacy, University of Illinois at Chicago, 601 S. Morgan St. (M/C 243), Chicago, IL, 60607, USA

    Darold T. Barnum

  2. College of Medicine, University of Illinois at Chicago, 1919 West Taylor Street, Room 149, Chicago, IL, 60612, USA

    Matthew Johnson

  3. College of Business Administration (Emeritus), Creighton University, Omaha, NE, 68178, USA

    John M. Gleason

Authors
  1. Darold T. Barnum
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthew Johnson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. John M. Gleason
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Darold T. Barnum.

Additional information

This article is part of the Topical Collection on Systems-Level Quality Improvement.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Barnum, D.T., Johnson, M. & Gleason, J.M. Importance of Statistical Evidence in Estimating Valid DEA Scores. J Med Syst 40, 47 (2016). https://doi.org/10.1007/s10916-015-0408-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-015-0408-y

Keywords

Search

Navigation