Abstract
The identification of environmental factors that explain differences in efficiency is essential for improving the results of public universities. A two-stage, semi-parametric approach with the single and double bootstrap procedure (Algorithm #1 and Algorithm #2) proposed by Simar and Wilson (J Econom 136(1):31–64, 2007) was used in this article for making valid inferences about the impact of environmental factors on university efficiency. A data envelopment analysis (DEA) efficiency estimator was used in the first stage to estimate technical efficiency scores for Spanish public universities. It is common to explore the determinants of (in)efficiency in a second stage. To provide valid inference, Simar and Wilson (2007) suggested a parametric bootstrap of the truncated regression (Algorithm #1). Alternatively, they recommended a bootstrap procedure to obtain bias-corrected technical efficiency scores used in the second-stage truncated regression; valid inference can be obtained by using a second bootstrap procedure applied to the truncated regression (Algorithm #2). Under both algorithms, three environmental factors were statistically significant predictors of efficiency. Our results confirmed that universities with a higher percentage of academics with tenure, outgoing Erasmus students, and state grantees tend to be less inefficient.
Similar content being viewed by others
Notes
Environmental variables are typically factors not under the control of the manager, but they may influence the production process. Environmental variables may influence the efficiency scores only or influencing the border of the production possibility set and that way also influencing the scores.
A one-stage procedure includes these variables in a single DEA stage (e.g., Daraio and Simar 2005).
It is also well known that the discrimination power of DEA will be much weakened if too many input or output indicators are used. The problem that arises is that, in DEA, the degrees of freedom increase with the number of DMUs and decrease with the number of inputs and outputs (Cooper et al. 2000).
Despite criticisms of this approach, as we will see later in this article, S&W's work is highly cited (2,653 citations as of October 20, 2019, according to Google Scholar).
Around 30% of Spanish undergraduates get a state grant, so they do not have to pay any tuition fees.
Spanish public universities do not have an amount of money from the public budget allocated for research.
Kuosmanen and Johnson (2010) led to the full integration of DEA and SFA into a unified framework of productivity analysis, referred to as stochastic nonparametric envelopment of data (StoNED).
In the analysis, the author showed that parametric methods provide lower estimates of efficiency than non-parametric methods (Johnes 2014).
Nonetheless, a weakness of DEA is that it is deterministic and attributes all deviations from the frontier to inefficiencies.
In our study, DMUs are universities or HEIs.
The first use of LP was in Farrell and Fieldhouse (1962).
HEIs are mainly funded by public funds. It seems reasonable to assume that the objective of the universities is oriented towards making the best use of available resources.
The merit of this technique has been acknowledged in recent studies (e.g., Chang et al. 2017).
We assume that all elements of z are continuous.
Conceivably, the environmental variables might affect only the distribution of efficiency among DMUs. The “separability" condition should be tested before estimating a second-stage regression but, until now, no test has been available (Daraio et al. 2016).
DEA produces a measure of efficiency relative to that achieved by the other producers or DMUs in the sample.
The model introduced by Banker and Natarajan (2008) did not impose separability, but it imposed other restrictive conditions that are not likely to be satisfied by real data. Actually, the discussion between Banker and Simar-Wilson has not finished yet.
Unfortunately, CRUE stopped providing researchers with this detailed information, so we do not have more up-to-date figures. Anyhow, the case of Spain is used to illustrate the proposed methodology.
Traditional campus-based public universities under the same legislation (Organic Law 6/2001, of December 21, on Universities), which are financed primarily with money from the public budget.
We had no information to measure the so-called third mission of universities.
Only traditional universities. The indicator is calculated as the percentage of students that do not matriculate at a university during the two following years.
Research income refers to the money that arrives at universities from competitive research projects in regional, state, and European open calls.
In the Spanish higher education system, students take courses in two semesters (each course or subject has six credits on average, about 4 h of class per week). On average, the course load in one academic year is 60 credits. All degrees of Diplomatura and Licenciatura were included.
In this paper, we consider the contribution of a variable to the total efficiency as determined by its level of input (or output) times the weight. See Angulo-Meza and Lins (2002) for further details.
As of July 31, 2008.
Research quality is positively related to teaching quality (Cadez et al., 2017).
Grants from the Spanish Ministry of Education, 2008/2009 academic year.
On a maximum score of 10 points.
Having a good academic record is one of the main requirements to participate in the Erasmus program.
U. Pompeu Fabra, U. de Lleida, U. Autónoma de Barcelona, U. Politécnica de Cataluña, and U. de Barcelona.
U. de La Laguna and U. de Las Palmas de Gran Canaria.
References
Abbott, M., & Doucouliagos, C. (2003). The efficiency of Australian universities: A data envelopment analysis. Economics of Education Review,22(1), 89–97.
Agasisti, T., & Dal Bianco, A. (2009). Reforming the university sector: Effects on teaching efficiency—evidence from Italy. Higher Education,57(4), 477–498.
Agasisti, T., Dal Bianco, A., Landoni, P., Sala, A., & Salerno, M. (2011). Evaluating the efficiency of research in academic departments: An empirical analysis in an Italian region. Higher Education Quarterly,65(3), 267–289.
Agasisti, T., & Pérez-Esparrells, C. (2010). Comparing efficiency in a cross-country perspective: The case of Italian and Spanish state universities. Higher Education,59(1), 85–103.
Agasisti, T., & Wolszczak-Derlacz, J. (2016). Exploring efficiency differentials between Italian and Polish universities, 2001–2011. Science and Public Policy,43(1), 128–142.
Ahn, T., Arnold, V., Charnes, A., & Cooper, W. W. (1989). DEA and ratio efficiency analyses for public institutions of higher learning in Texas. Research in Governmental and Nonprofit Accounting,5, 165–185.
Aigner, D., Lovell, C. K., & Schmidt, P. (1977). Formulation and estimation of stochastic frontier production function models. Journal of Econometrics,6(1), 21–37.
Angulo-Meza, L., & Lins, M. P. E. (2002). Review of methods for increasing discrimination in data envelopment analysis. Annals of Operations Research,116(1–4), 225–242.
Athanassopoulos, A. D., & Shale, E. (1997). Assessing the comparative efficiency of higher education institutions in the UK by means of data envelopment analysis. Education Economics,5(2), 117–134.
Avkiran, N. K. (2001). Investigating technical and scale efficiencies of Australian universities through data envelopment analysis. Socio-Economic Planning Sciences,35, 57–80.
Badunenko, O., & Mozharovskyi, P. (2016). Nonparametric frontier analysis using Stata. The Stata Journal,16(3), 550–589.
Badunenko, O., & Tauchmann, H. (2017). ‘SIMARWILSON’: Module to perform Simar and Wilson (2007) efficiency analysis. Statistical Software Components. Retrieved from https://ideas.repec.org/c/boc/bocode/s458156.html.
Banker, R. D., Charnes, A., & Cooper, W. W. (1984). Some models for estimating technical and scale inefficiencies in data envelopment analysis. Management Science,30(9), 1078–1092.
Banker, R. D., & Morey, R. C. (1986). Efficiency analysis for exogenously fixed inputs and outputs. Operations Research,34(4), 513–521.
Banker, R. D., & Natarajan, R. (2008). Evaluating contextual variables affecting productivity using data envelopment analysis. Operations Research,56(1), 48–58.
Banker, R. D., Natarajan, R., & Zhang, D. (2015). Estimation of the impact of contextual variables in stochastic frontier production function models using data envelopment analysis: Two-stage versus bootstrap approaches. Temple University Working Paper.
Beasley, J. (1995). Determining teaching and research efficiencies. Journal of the Operational Research Society,46, 441–452.
Berbegal-Mirabent, J., Lafuente, E., & Solé, F. (2013). The pursuit of knowledge transfer activities: An efficiency analysis of Spanish universities. Journal of Business Research,66(10), 2051–2059.
Bess, J. L. (1998). Contract systems, bureaucracies, and faculty motivation: The probable effects of a no-tenure policy. Journal of Higher Education,69(1), 1–22.
Bougnol, M. L., & Dulá, J. H. (2006). Validating DEA as a ranking tool: An application of DEA to assess performance in higher education. Annals of Operations Research,145(1), 339–365.
Bradley, S., Johnes, G., & Millington, J. (2001). The effect of competition on the efficiency of secondary schools in England. European Journal of Operational Research,135(3), 545–568.
Breu, T. M., & Raab, R. L. (1994). Efficiency and perceived quality of the nation’s “top 25” National Universities and National Liberal Arts Colleges: An application of data envelopment analysis to higher education. Socio-Economic Planning Sciences,28, 33–45.
Cadez, S., Dimovski, V., & Zaman Groff, M. (2017). Research, teaching and performance evaluation in academia: The salience of quality. Studies in Higher Education,42(8), 1455–1473.
Caldera, A., & Debande, O. (2010). Performance of Spanish universities in technology transfer: An empirical analysis. Research Policy,39(9), 1160–1173.
Chang, Y. T., Lee, S., & Park, H. K. (2017). Efficiency analysis of major cruise lines. Tourism Management,58, 78–88.
Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision-making units. European Journal of Operational Research,2, 429–444.
Coelli, T. J., Rao, D. S. P., O’Donnell, C. J., & Battese, G. E. (2005). An introduction to efficiency and productivity analysis (2nd ed.). Berlin: Springer.
Cohn, E., Rhine, S., & Santos, M. (1989). Institutions of higher education as multi-product firms: Economies of scale and scope. Review of Economics and Statistics,71(2), 284–290.
Cooper, W. W., Seiford, L. M., & Tone, K. (2000). Data envelopment analysis: A comprehensive text with models, applications, references and DEA-Solver Software. Boston: Kluwer Academic Publishers.
Cooper, W. W., Seiford, L. M., & Zhu, J. (2011). Handbook on data envelopment analysis. New York: Springer.
Daraio, C., & Simar, L. (2005). Introducing environmental variables in nonparametric frontier models: A probabilistic approach. Journal of Productivity Analysis,24(1), 93–121.
Daraio, C., Simar, L., & Wilson, P. W. (2016). Nonparametric estimation of efficiency in the presence of environmental variables. Department of Computer, Control and Management Engineering Technical Report No. 2016-02, Sapienza-Universitá di Roma.
De la Torre, E. M., Agasisti, T., & Perez-Esparrells, C. (2017a). The relevance of knowledge transfer for universities’ efficiency scores: An empirical approximation on the Spanish public higher education system. Research Evaluation,26(3), 211–229.
De la Torre, E. M., Gómez-Sancho, J. M., & Perez-Esparrells, C. (2017b). Comparing university performance by legal status: A Malmquist-type index approach for the case of the Spanish higher education system. Tertiary Education and Management,23(3), 206–221.
De Witte, K., & López-Torres, L. (2017). Efficiency in education: A review of literature and a way forward. Journal of the Operational Research Society,68(4), 339–363.
Debreu, G. (1951). The coefficient of resource utilization. Econometrica,19(3), 273–292.
Dynarski, S. M. (2003). Does aid matter? Measuring the effect of student aid on college attendance and completion. American Economic Review,93(1), 279–288.
Edvardsen, D. F., Førsund, F. R., & Kittelsen, S. A. (2017). Productivity development of Norwegian institutions of higher education 2004–2013. Journal of the Operational Research Society,68(4), 399–415.
Emrouznejad, A., Parker, B. R., & Tavares, G. (2008). Evaluation of research in efficiency and productivity: A survey and analysis of the first 30 years of scholarly literature in DEA. Journal of Socio-Economics Planning Science,42(3), 151–157.
Estes, B., & Polnick, B. (2012). Examining motivation theory in higher education: An expectancy theory analysis of tenured faculty productivity. International Journal of Management, Business, and Administration,15(1), 1–7.
Fandel, G. (2007). On the performance of universities in North Rhine-Westphalia, Germany: Government’s redistribution of funds judged using DEA efficiency measures. European Journal of Operational Research,176, 521–533.
Färe, R., Grosskopf, S., & Lovell, C. A. K. (1985). The measurement of efficiency of production. Boston, MA: Kluwer-Nijhoff Publishing.
Färe, R., Grosskopf, S., Norris, M., & Zhongyang, Z. (1994). Productivity growth, technical progress and efficiency change in industrialized countries. American Economic Review,84(1), 66–83.
Färe, R., & Primont, D. (1995). Multi-output production and duality: Theory and applications. Boston: Kluwer Academic Publishers.
Faria, J. R., & McAdam, P. (2014). Does tenure make researchers less productive? The case of the “specialist”. University of Surrey Discussion Papers in Economics No. 5.
Farrell, M. J. (1957). The measurement of productive efficiency. Journal of the Royal Statistical Society: Series A (General),120(3), 253–281.
Farrell, M. J., & Fieldhouse, M. (1962). Estimating efficient production functions under increasing returns to scale. Journal of the Royal Statistical Society: Series A (General),125(2), 252–267.
Flegg, T., Allen, D., Field, K., & Thurlow, T. W. (2004). Measuring the efficiency of British universities: A multi-period data envelopment analysis. Education Economics,12(3), 231–249.
Fried, H. O., Lovell, C. A. K., & Schmidt, S. S. (Eds.). (2008). The measurement of productive efficiency and productivity growth. Oxford: Oxford University Press.
Gillen, D., & Lall, A. (1997). Developing measures of airport productivity and performance: An application of data envelopment analysis. Transportation Research Part E: Logistics and Transportation Review,33(4), 261–273.
Gralka, S., Wohlrabe, K., & Bornmann, L. (2019). How to measure research efficiency in higher education? Research grants vs. publication output. Journal of Higher Education Policy and Management,41(3), 322–341.
Halkos, G., Tzeremes, N. G., & Kourtzidis, S. A. (2012). Measuring public owned university departments’ efficiency: A bootstrapped DEA approach. Journal of Economics and Econometrics,55(2), 1–24.
Johnes, J. (2006a). Data envelopment analysis and its application to the measurement of efficiency in higher education. Economics of Education Review,25(3), 273–288.
Johnes, J. (2006b). Measuring teaching efficiency in higher education: An application of data envelopment analysis to economics graduates from UK universities 1993. European Journal of Operational Research,174, 443–456.
Johnes, J. (2014). Efficiency and mergers in English higher education 1996/97 to 2008/9: Parametric and non-parametric estimation of the multi-input multi-output distance function. Manchester School,82(4), 465–487.
Johnes, J., & Johnes, G. (1995). Research funding and performance in U.K. university departments of economics: A frontier analysis. Economics of Education Review,14, 301–314.
Johnes, G., & Johnes, J. (2016). Costs, efficiency, and economies of scale and scope in the English higher education sector. Oxford Review of Economic Policy,32(4), 596–614.
Johnes, J., & Yu, L. (2008). Measuring the research performance of Chinese higher education institutions using data envelopment analysis. China Economic Review,19, 679–696.
Kao, C., & Hung, H. T. (2008). Efficiency analysis of university departments: An empirical study. Omega,36(4), 653–664.
Kivistö, J., Pekkola, E., Berg, L. N., Hansen, H. F., Geschwind, L., & Lyytinen, A. (2019). Performance in higher education institutions and its variations in Nordic policy. In R. Pinheiro, L. Geschwind, H. F. Hansen, & K. Pulkkinen (Eds.), Reforms, organizational change and performance in higher education: A comparative account from the Nordic Countries (pp. 37–67). Cham: Palgrave Macmillan.
Koopmans, T. C. (1951). Analysis of production as an efficient combination of activities. In T. C. Koopmans (Ed.), Activity analysis of production and allocation. Cowles Commission for Research in Economics: Monograph No. 13 (pp. 33–97). New York: Wiley.
Kuosmanen, T., & Johnson, A. L. (2010). Data envelopment analysis as nonparametric least-squares regression. Operations Research,58(1), 149–160.
Lassibille, G., & Navarro-Gómez, M. L. (2011). How long does it take to earn a higher education degree in Spain? Research in Higher Education,52(1), 63–80.
Lesjak, M., Juvan, E., Ineson, E. M., Yap, M. H., & Axelsson, E. P. (2015). Erasmus student motivation: Why and where to go? Higher Education,70(5), 845–865.
Liu, J. S., Lu, L. Y. Y., Lu, W. M., & Lin, B. J. Y. (2013). A survey of DEA applications. Omega,41, 893–902.
Lovell, C. A. K. (1993). Production frontiers and productive efficiency. In H. O. Fried, C. A. K. Lovell, & S. S. Schmidt (Eds.), The measurement of productive efficiency: Techniques and applications (pp. 3–67). Oxford: Oxford University Press.
McMillan, M. L., & Datta, D. (1998). The relative efficiencies of Canadian universities: A DEA perspective. Canadian Public Policy/Analyse de Politiques,24(4), 485–511.
MECD (2014): Datos básicos del sistema universitario español. Curso 2013-2014, Ministerio de Educación, Cultura y Deporte, Madrid.
Meeusen, W., & van den Broeck, J. (1977). Efficiency estimation from Cobb–Douglas production functions with composed error. International Economic Review,18(2), 435–444.
Muñiz, M. A. (2002). Separating managerial inefficiency and external conditions in data envelopment analysis. European Journal of Operational Research,143(3), 625–643.
Nazarko, J., & Šaparauskas, J. (2014). Application of DEA method in efficiency evaluation of public higher education institutions. Technological and Economic Development of Economy,20(1), 25–44.
Neely, A., Gregory, M., & Platts, K. (1995). Performance measurement system design: A literature review and research agenda. International Journal of Operations and Production Management,15(4), 80–116.
Ng, Y. C., & Li, S. K. (2000). Measuring the research performance of Chinese higher education institutions: An application of data envelopment analysis. Education Economics,8(2), 139–156.
OECD. (2017). Benchmarking higher education system performance: Conceptual framework and data. Enhancing higher education system performance. Paris: OECD.
Orea, L., & Zofío, J. L. (2017). A primer on the theory and practice of efficiency and productivity analysis. University of Oviedo Efficiency Series Paper No. 5.
Ray, S. C. (1988). Data envelopment analysis, nondiscretionary inputs and efficiency: An alternative interpretation. Socio-Economic Planning Sciences,22(4), 167–176.
Rørstad, K., & Aksnes, D. W. (2015). Publication rate expressed by age, gender and academic position—A large-scale analysis of Norwegian academic staff. Journal of Informetrics,9(2), 317–333.
Santelices, M. V., Catalán, X., Kruger, D., & Horn, C. (2016). Determinants of persistence and the role of financial aid: Lessons from Chile. Higher Education,71(3), 323–342.
Seiford, L. M. (1997). A bibliography for data envelopment analysis (1978–1996). Annals of Operations Research,73, 393–438.
Simar, L., & Wilson, P. W. (1998). Sensitivity analysis of efficiency scores: How to bootstrap in nonparametric frontier models. Management Science,44, 49–61.
Simar, L., & Wilson, P. W. (2000). A general methodology for bootstrapping in non-parametric frontier models. Journal of Applied Statistics,27(6), 779–802.
Simar, L., & Wilson, P. W. (2007). Estimation and inference in two-stage, semi-parametric models of production processes. Journal of Econometrics,136(1), 31–64.
Simar, L., & Wilson, P. W. (2008). Statistical interference in nonparametric frontier models: Recent developments and perspectives. In H. O. Fried, C. A. Knox Lovell, & S. S. Schmidt (Eds.), The measurement of productive efficiency and productivity growth (pp. 421–521). Oxford: Oxford University Press.
Simar, L., & Wilson, P. W. (2015). Statistical approaches for non-parametric frontier models: A guided tour. International Statistical Review,83(1), 77–110.
Stern, Z. S., Mehrez, A., & Barboy, A. (1994). Academic departments efficiency via DEA. Computers and Operations Research,21, 543–556.
Thanassoulis, E., Portela, M. C. S., & Despić, O. (2008). Data envelopment analysis: The mathematical programming approach to efficiency analysis. In H. O. Fried, C. A. K. Lovell, & S. S. Schmidt (Eds.), The measurement of productive efficiency and productivity growth (pp. 251–420). Oxford: Oxford University Press.
Titus, M. A., & Eagan, K. (2016). Examining production efficiency in higher education: The utility of stochastic frontier analysis. In M. Paulsen (Ed.), Higher education: Handbook of theory and research (Vol. 31, pp. 441–512). Berlin: Springer.
Tyagi, P., Yadav, S. P., & Singh, S. P. (2009). Relative performance of academic departments using DEA with sensitivity analysis. Evaluation and Program Planning,32(2), 168–177.
Tziogkidis, P. (2012). Bootstrap DEA and hypothesis testing (No. E2012/18). Cardiff Economics Working Papers.
Warning, S. (2004). Performance differences in German higher education: Empirical analysis of strategic groups. Review of Industrial Organization,24(4), 393–408.
Wolszczak-Derlacz, J., & Parteka, A. (2011). Efficiency of European public higher education institutions: A two-stage multicountry approach. Scientometrics,89(3), 887–917.
Worthington, A. (2001). An empirical survey of frontier efficiency measurement techniques in education. Education Economics,9(3), 245–268.
Zhu, J. (Ed.). (2016). Data envelopment analysis: A handbook of empirical studies and applications. New York: Springer.
Author information
Authors and Affiliations
Corresponding author
Additional information
The author highly appreciates the valuable comments of the referee on previous versions of this article. These comments helped me to improve the manuscript significantly.
Appendix
Appendix
See Table 3.
Rights and permissions
About this article
Cite this article
Salas-Velasco, M. Measuring and explaining the production efficiency of Spanish universities using a non-parametric approach and a bootstrapped-truncated regression. Scientometrics 122, 825–846 (2020). https://doi.org/10.1007/s11192-019-03324-4
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-019-03324-4
Keywords
- Data envelopment analysis
- Bootstrapped-truncated regression
- Simar and Wilson
- Efficiency measurement
- Spanish public universities