Skip to main content
SpringerLink
Log in

Efficiency assessment and convergence in teaching and research in Italian public universities

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

This paper investigates the pattern of teaching and research performances, the relationship between them, and the convergence for Italian public HEIs in the period 2000–2010, by comparing different bootstrap robust non-parametric frontier estimators. Overall we find an efficiency improvement, mainly driven by research, whereas teaching efficiency increases only in the very first years of the sample period. We also ascertain a slightly positive relationship between research and teaching performances. Furthermore, we find that Italian HEIs converge, in the observed period, although research and teaching do it at a different pace. Our empirical findings are robust to alternative estimators and bootstrapped bias correction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. They were originally designed for development economics and recently applied to the study of the dynamics of technical efficiency of banks and financial institutions (Weill 2009; Casu and Girardone 2010; Zhang and Matthews 2012; Ayadi et al. 2013), and teaching in HEIs (Guccio et al. 2015).

  2. An increased interest in cross-country comparisons emerged in more recent times, due to the reform of HEI systems within EU (the so-called Bologna Process) that aimed at creating a common European HEI system. An example of such a strand of research is Parteka and Wolszczak-Derlacz (2013) that compares 266 HEIs in 7 European countries in the period 2001–2005.

  3. The advantage of using a non-parametric technique is that it provides a “best performance” frontier that identifies efficient HEIs able to provide a benchmark for under-performing HEIs. An alternative method identifying a frontier is the Stochastic Frontier Analysis (SFA; Aigner et al. 1977; Meeusen and Van den Broeck 1977). In contrast to non-parametric frontier, this approach provides a measure of statistical noise in its estimation of efficiency. However, SFA requires the specification of the functional form of the production process and the distribution of data and error terms.

  4. Notwithstanding their large use, traditional non-parametric models such as DEA and FDH estimators have received some criticism since they rely on extreme points, and they could be extremely sensitive to data selection, aggregation, model specification and data errors (Simar and Wilson 2008). As an alternative, estimators based on partial frontiers and the resulting partial efficiency scores are proposed to provide robust measures of efficiency at extreme data points: Cazals et al. (2002) proposes the nonparametric order-m estimator and Daouia and Simar (2007) order-α quantile type frontiers.

  5. According to Badin and Simar (2009), this estimator is a symmetric version of the order-m minimum input function proposed by Cazals et al. (2002).

  6. Although scope economies generally refer to decreasing average costs in producing two or more outputs, in this paper we refer to the benefits that teaching may have from research and vice versa, in line with Bonaccorsi et al. (2006, 2007) and De Witte et al. (2013b).

  7. An alternative to analyze the marginal effect of one activity (assumed to be an external or environmental factor) on the other one’s performance is to employ a two-stage DEA analysis (i.e. Sellers-Rubio et al. 2010). A robust semiparametric approach proposed by Simar and Wilson (2007) requires a restrictive separability condition between the input–output space and the space of external or environmental factors. This is certainly unrealistic in our analysis. We thank an anonymous reviewer for this point.

  8. Notice that the ratios are not bounded by 1, in case of m-order estimator.

  9. Agasisti and Dal Bianco (2009) and Guccio et al. (2015), for instance, consider as input measure the number of enrolments with a score equal or greater than 9/10 in secondary school.

  10. The sample is coherent with the one used by Agasisti and Wolszczak-Derlacz (2014) and virtually represents all the public HEIs with general purpose in Italian system. Some public HEIs are excluded due the incompleteness of the data in the observed period (namely: Università di Macerata; Politecnico di Bari; Università di Napoli “L’Orientale”; IUAV di Venezia). Other HEIs are excluded from the sample for the peculiarities of those institutions (Scuola Normale Superiore di Pisa; Scuola Superiore di Studi Universitari e Perfezionamento Sant'Anna di Pisa; Scuola Internazionale Superiore di Studi Avanzati di Trieste; Scuola IMT di Lucca; Istituto Universitario di Studi Superiori di Pavia; Università degli Studi di Roma "Foro Italico"; Università per Stranieri di Siena; Università per Stranieri di Perugia).

  11. Results are robust with respect to bias correction and to the assumptions of variable returns to scale and free disposability of inputs, and results are available upon request.

  12. STUD stands for STUD• 104.

  13. GRAD stands for GRAD• 103.

  14. RESEARCH stands for RESEARCH• 103.

  15. To save space we do not report all the results obtained using different estimators. The analytical results of the conditional and unconditional estimates are available upon request.

  16. We also perform a number of other robustness checks to assess effects between other activities and performances including a two-stage robust semiparametric approach proposed by Simar and Wilson (2007) obtaining results comparable with those reported here. Results are available upon request.

  17. However, our findings are robust with respect to variable returns to scale. Estimates are available upon request.

References

  • Abbott, M., & Doucouliagos, C. (2003). The efficiency of Australian universities: A data envelopment analysis. Economics of Education Review, 22, 89.

    Article  Google Scholar 

  • Agasisti, T., & Catalano, G. (2006). Governance models of university systems towards quasi-markets? European comparison. Journal of Higher Education policy and Management, 28, 245–262.

    Article  Google Scholar 

  • Agasisti, T., & Dal Bianco, A. (2009). Reforming the university sector: Effects on teaching efficiency-evidence from Italy. Higher Education, 57, 477–498.

    Article  Google Scholar 

  • Agasisti, T., & Pérez-Esparrells, C. (2010). Comparing efficiency in a cross-country perspective: The case of Italian and Spanish state universities. Higher Education, 59, 85–103.

    Article  Google Scholar 

  • Agasisti, T., & Pohl, C. (2012). Comparing German and Italian public universities: Convergence or divergence in the higher education landscape? Managerial and Decision Economics, 33, 71–85.

    Article  Google Scholar 

  • Agasisti, T., & Wolszczak-Derlacz, J. (2014). Exploring universities’ efficiency differentials between countries in a multi-year perspective: An application of bootstrap DEA and Malmquist index to Italy and Poland, 2001–2011, IRLE working paper no. 113-14.

  • Aghion, P., Dewatripont, M., Hoxby, C., Mas-Colell, A., & Sapir, A. (2010). The governance and performance of universities: Evidence from Europe and the US. Economic Policy, 25, 7–59.

    Article  Google Scholar 

  • Aigner, D. J., Lovell, C. A. K., & Schmidt, P. (1977). Formulation and estimation of stochastic frontier production functions. Journal of Econometrics, 6, 21–37.

    Article  MathSciNet  MATH  Google Scholar 

  • Archibald, R. B., & Feldman, D. H. (2008). Explaining increases in higher education costs. Journal of Higher Education, 79, 268–295.

    Article  Google Scholar 

  • Athanassopoulos, A., & Shale, E. (1997). Assessing the comparative efficiency of higher education institutions in the UK by means of data envelopment analysis. Education Economics, 5, 117–134.

    Article  Google Scholar 

  • Auranen, O., & Nieminenb, M. (2010). University research funding and publication performance—An international comparison. Research Policy, 39, 822–834.

    Article  Google Scholar 

  • Avkiran, N. K. (2001). Investigating technical and scale efficiencies of Australian universities through data envelopment analysis. Socio-Economic Planning Sciences, 35, 57–80.

    Article  Google Scholar 

  • Ayadi, R., Boussemart, J., Leleu, H., & Saidane, D. (2013). Mergers and acquisitions in European banking higher productivity or better synergy among business lines? Journal of Productivity Analysis, 39, 165–175.

    Article  Google Scholar 

  • Badin, L., & Simar, L. (2009). A bias-corrected nonparametric envelopment estimator of frontiers. Econometric Theory, 25, 1289–1318.

    Article  MathSciNet  MATH  Google Scholar 

  • Barro, R. J., & Sala-i-Martin, X. (1991). Convergence across states and regions. Brookings Papers Economic Activity, 1, 107–182.

    Article  Google Scholar 

  • Barro, R. J., & Sala-i-Martin, X. (1992). Convergence. Journal of Political Economy, 100, 223–251.

    Article  Google Scholar 

  • Beasley, J. (1990). Comparing university departments. Omega, 18, 171–183.

    Article  Google Scholar 

  • Beasley, J. (1995). Determining teaching and research efficiencies. Journal of the Operational Research Society, 46, 441–452.

    Article  MATH  Google Scholar 

  • Bergantino, A. S., Capozza, C., & Porcelli, F. (2013). Measuring the efficiency of the Italian university system: The role of market structure. A two step DEA analysis at faculty level. Paper presented at the XXV conference of the Italian Association of Public Economics (SIEP).

  • Bonaccorsi, A., Daraio, C., & Simar, L. (2006). Advanced indicators of productivity of universities an application of robust nonparametric methods to Italian data. Scientometrics, 66, 389–410.

    Article  Google Scholar 

  • Bonaccorsi, A., Daraio, C., & Simar, L. (2007). Productivity and efficiency of European universities. In A. Bonaccorsi & C. Daraio (Eds.), Universities and strategic knowledge creation: Specialization and performance in Europe. Cheltenham: Edward Elgar.

    Chapter  Google Scholar 

  • Brunori, P., Peragine, V., & Serlenga, L. (2012). Fairness in education: The Italian university before and after the reform. Economics of Education Review, 31, 764–777.

    Article  Google Scholar 

  • Cappellari, L., & Lucifora, C. (2009). The “Bologna Process” and college enrollment decisions. Labour Economics, 16, 638–647.

    Article  Google Scholar 

  • Casu, B., & Girardone, C. (2010). Integration and efficiency convergence in EU banking markets. Omega, 38, 260–267.

    Article  Google Scholar 

  • Cazals, C., Florens, J. P., & Simar, L. (2002). Nonparametric frontier estimation: A robust approach. Journal of Econometrics, 106, 1–25.

    Article  MathSciNet  MATH  Google Scholar 

  • Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision making units. European Journal of Operational Research, 2, 429–444.

    Article  MathSciNet  MATH  Google Scholar 

  • Daouia, A., & Simar, L. (2007). Non-parametric efficiency analysis: A multivariate conditional quantile approach. Journal of Econometrics, 140, 375–400.

    Article  MathSciNet  MATH  Google Scholar 

  • Daraio, C., & Simar, L. (2005). Introducing environmental variables in nonparametric frontier models: A probabilistic approach. Journal of Productivity Analysis, 24, 93–121.

    Article  Google Scholar 

  • Daraio, C., & Simar, L. (2007). Advanced robust and nonparametric methods in efficiency analysis. Methodology and applications. New York: Springer.

    MATH  Google Scholar 

  • De Witte, K., Rogge, N., Cherchye, L., & Van Puyenbroeck, T. (2013a). Accounting for economies of scope in performance evaluations of university professors. Journal of the Operational Research Society, 64, 1595–1606.

    Article  Google Scholar 

  • De Witte, K., Rogge, N., Cherchye, L., & Van Puyenbroeck, T. (2013b). Economies of scope in research and teaching: A non-parametric investigation. Omega, 41, 305–314.

    Article  Google Scholar 

  • Debreu, G. (1951). The coefficient of resource utilization. Econometrica, 19, 273–292.

    Article  MATH  Google Scholar 

  • Deprins, D., Simar, L., & Tulkens, H. (1984). Measuring labor efficiency in post offices. In M. Marchand, P. Pestieau, & H. Tulkens (Eds.), The performance of public enterprises: Concepts and measurements. Amsterdam: North Holland.

    Google Scholar 

  • Di Pietro, G., & Cutillo, A. (2008). Degree flexibility and university drop-out: The Italian experience. Economics of Education Review, 27, 546–555.

    Article  Google Scholar 

  • Dundar, H., & Lewis, D. (1995). Departmental productivity in American universities: Economies of scale and scope. Economics of Education Review, 14, 119–144.

    Article  Google Scholar 

  • Fare, R., Grosskopf, S., & Lovell, C. (1994). Production frontiers. Cambridge: Cambridge University Press.

    Google Scholar 

  • Farrell, M. (1957). The measurement of productive efficiency. Journal of the Royal Statistical Society Series A (General), 120, 253–290.

    Article  Google Scholar 

  • Feldman, K. (1987). Exploring the relationship between research productivity and college students’ ratings of teachers: Quantitative reviews and the logic of analysis. Paper presented at the Annual Meeting of the American Educational Research Association, Washington, DC.

  • Flegg, A. T., Allen, D. O., Field, K., & Thurlow, T. W. (2004). Measuring the efficiency of British universities: A multi-period data envelopment analysis. Education Economics, 12, 231–249.

    Article  Google Scholar 

  • Guccio, C., Martorana, M. F., & Monaco, L. (2015). Evaluating the impact of the Bologna Process on the efficiency convergence of Italian universities: A non-parametric frontier approach. Journal of Productivity Analysis. doi:10.1007/s11123-015-0459.

    Google Scholar 

  • Hattie, J., & Marsh, H. W. (1996). The relationship between research and teaching: A meta-analysis. Review of Educational Research, 66, 507–542.

    Article  Google Scholar 

  • Hattie, J., & Marsh, H. W. (2004). One journey to unravel the relationship between research and teaching. Research and teaching: Closing the divide? An international colloquium, Winchester, March 18–19.

  • Herbst, M. (2007). Financing public universities: The case of performance funding. Dordrecht: Springer.

    Google Scholar 

  • Hicks, D. (2012). Performance-based university research funding systems. Research Policy, 41, 251–261.

    Article  Google Scholar 

  • Huisman, J., & Van der Wende, M. (2004). The EU and Bologna: Are supra-and international initiatives threatening domestic agendas? European Journal of Education, 39, 349–357.

    Article  Google Scholar 

  • Johnes, J. (2006). Data envelopment analysis and its applications to the measurement of efficiency in higher education. Economics of Education Review, 25, 273–288.

    Article  MATH  Google Scholar 

  • Johnes, J. (2008). Efficiency and productivity change in the English higher education sector from 1996/97 to 2004/5. Manchester School, 76, 653–674.

    Article  Google Scholar 

  • Johnes, J., & Johnes, G. (1995). Research funding and performance in U.K. University departments of economics: A frontier analysis. Economics of Education Review, 14, 301–314.

    Article  Google Scholar 

  • Johnes, J., & Yu, L. (2008). Measuring the research performance of Chinese higher education institutions using data envelopment analysis. China Economic Review, 19, 679–696.

    Article  Google Scholar 

  • Joumady, O., & Ris, C. (2005). Performance in European higher education: A non-parametric production frontier approach. Education Economics, 13, 189–205.

    Article  Google Scholar 

  • Kantabutra, S., & Tang, J. (2010). Efficiency analysis of public universities in Thailand. Tertiary Education and Management, 16, 15–33.

    Article  Google Scholar 

  • Kempkes, G., & Pohl, C. (2010). The efficiency of German universities—Some evidence from nonparametric and parametric methods. Applied Economics, 42, 2063–2079.

    Article  Google Scholar 

  • Koopmans, T. C. (1951). An analysis of production as an efficient combination of activities. In T. C. Koopmans (Ed.), Activity analysis of production and allocation, Cowles Commission for Research in Economics, monograph 13 (pp. 33–37). New York: Wiley.

    Google Scholar 

  • Kremer, J. (1990). Construct validity of multiple measures in teaching, research and services and reliability of peer ratings. Journal of Educational Psychology, 82, 213–218.

    Article  Google Scholar 

  • Kuah, C., & Wong, K. (2011). Efficiency assessment of universities through data envelopment analysis. Procedia Computer Science, 3, 499–506.

    Article  Google Scholar 

  • Lambert, R., & Butler, N. (2006). The future of European universities: Renaissance or decay?. London: Centre for European Reform.

    Google Scholar 

  • Laureti, T., Secondi, L., & Biggeri, L. (2014). Measuring the efficiency of teaching activities in Italian universities: An information theoretic approach. Economics of Education Review, 42, 147–164.

    Article  Google Scholar 

  • Madden, G., Savage, S., & Kemp, S. (1997). Measuring public sector efficiency: A study of economics departments at Australian universities. Education Economics, 5, 153–168.

    Article  Google Scholar 

  • Meeusen, W., & van den Broeck, J. (1977). Efficiency estimation from Cobb–Douglas production function with composed error. International Economic Review, 8, 435–444.

    Article  MATH  Google Scholar 

  • Neumann, R. (1992). Perceptions of the teaching-research nexus: A framework for analysis. Higher Education, 23, 159–171.

    Article  Google Scholar 

  • Olivares, M., & Wetzel, H. (2011). Competing in the higher education market: Empirical evidence for economies of scale and scope in German higher education institutions, University of Lüneburg working paper series in economics, no. 223.

  • Parteka, A., & Wolszczak-Derlacz, J. (2013). Dynamics of productivity in higher education: Cross-European evidence based on bootstrapped Malmquist indices. Journal of Productivity Analysis, 40, 67–82.

    Article  Google Scholar 

  • Rebora, G., & Turri, M. (2013). The UK and Italian research assessment exercises face to face. Research Policy, 42, 1657–1666.

    Article  Google Scholar 

  • Sánchez-Barrioluengo, M. (2014a). Articulating the ‘three-missions’ in Spanish universities. Research Policy, 43(10), 1760–1773.

    Article  Google Scholar 

  • Sav, G. T. (2012). Productivity, efficiency, and managerial performance regress and gains in United States universities: A data envelopment analysis. Advances in Management & Applied Economics, 2, 13–32.

    Google Scholar 

  • Sellers-Rubio, R., Mas-Ruiz, F. J., & Casado-Dìaz, A. B. (2010). University efficiency: Complementariness versus trade-off between teaching, research and administrative activities. Higher Education Quarterly, 64, 373–391.

    Article  Google Scholar 

  • Sheather, S. J., & Jones, M. C. (1991). A reliable data-based bandwidth selection method for Kernel density estimation. Journal of the Royal Statistical Society: Series B (Methodological), 53, 683–690.

    MathSciNet  MATH  Google Scholar 

  • Shephard, R. W. (1970). Theory of cost and production functions. Princeton: Princeton University Press.

    MATH  Google Scholar 

  • Simar, L., & Wilson, P. (1998). Sensitivity analysis of efficiency scores: How to bootstrap in nonparametric frontier models. Management Science, 44, 49–61.

    Article  MATH  Google Scholar 

  • Simar, L., & Wilson, P. (2000). Statistical inference in nonparametric frontier models: The state of the art. Journal of Productivity Analysis, 13, 49–78.

    Article  Google Scholar 

  • Simar, L., & Wilson, P. (2002). Nonparametric tests of returns to scale. European Journal of Operations Research, 139, 115–132.

    Article  MathSciNet  MATH  Google Scholar 

  • Simar, L., & Wilson, P. (2007). Estimation and inference in two-stage, semi-parametric models of production processes. Journal of Econometrics, 136, 31–64.

    Article  MathSciNet  MATH  Google Scholar 

  • Simar, L., & Wilson, P. (2008). Statistical inference in nonparametric frontier models: Recent developments and perspectives. In H. O. Fried, C. A. Knox Lovell, & S. S. Schmidt (Eds.), The measurement of productive efficiency and productivity growth (pp. 421–521). New York: Oxford University Press.

    Google Scholar 

  • St. Aubyn, M., Pina, A., Garcia, F., & Pais, J. (2009). Study on the efficiency and effectiveness of public spending on tertiary education. Directorate General Economic and Monetary Affairs (DG ECFIN), European Commission economic papers, no. 390.

  • Tomkins, C., & Green, R. (1988). An experiment in the use of data envelopment analysis for evaluating the efficiency of UK university departments of accounting. Financial Accountability And Management, 4, 147–164.

    Article  Google Scholar 

  • Wand, M. P., & Jones, M. C. (1995). Kernel smoothing. London: Chapman and Hall Press.

    Book  MATH  Google Scholar 

  • Weill, L. (2009). Convergence in banking efficiency across European countries. Journal of International Financial Markets, Institutions and Money, 19, 818–833.

    Article  Google Scholar 

  • Wolszczak-Derlacz, J., & Parteka, A. (2010). Scientific productivity of public higher education institutions in Poland—A comparative bibliometric analysis. EY—Better Governance, Warsaw.

  • Wolszczak-Derlacz, J., & Parteka, A. (2011). Efficiency of European public higher education institutions: A two-stage multicountry approach. Scientometrics, 89, 887–917.

    Article  Google Scholar 

  • Worthington, A. (2001). An empirical survey of frontier efficiency measurement techniques in education. Education Economics, 9, 245–268.

    Article  Google Scholar 

  • Zhang, T., & Matthews, K. (2012). Efficiency convergence properties of Indonesian banks 1992–2007. Applied Financial Economics, 22, 1465–1478.

    Article  Google Scholar 

Download references

Acknowledgments

Preliminary versions of this paper were presented in the XXVII annual congress of the Italian Society of Public Economics (Ferrara, 2015), and seminars at the Kyoto Sangyo University and University of Munich. We thank the participants and discussants, as well as two anonymous reviewers for several helpful comments. The usual caveat applies.

Author information

Authors and Affiliations

  1. Department of Economics and Business, University of Catania, Corso Italia, 55, 95129, Catania, Italy

    Calogero Guccio, Marco Ferdinando Martorana & Isidoro Mazza

Authors
  1. Calogero Guccio
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marco Ferdinando Martorana
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Isidoro Mazza
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Calogero Guccio.

Appendix

Appendix

See Tables 9, 10, 11 and 12.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guccio, C., Martorana, M.F. & Mazza, I. Efficiency assessment and convergence in teaching and research in Italian public universities. Scientometrics 107, 1063–1094 (2016). https://doi.org/10.1007/s11192-016-1903-8

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-016-1903-8

Keywords

JEL Classification

Search

Navigation