Skip to main content
SpringerLink
Log in

The determinants of the research output of universities: specialization, quality and inefficiencies

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

The analysis of productivity in Higher Education Institutions (HEIs) at a European level reveals enormous differences in output per researcher across countries. This study develops a 5-step methodology that explicitly considers the quality of scientific output in EU universities and its specialisations to explain and decompose the differences in output per university researcher in terms of (a) differences in efficiency within each field of science (FOS), (b) differences in FOS specialisation of HEIs in each country, (c) differences in quality, and (d) differences in allocation of resources per researcher. The inefficiency levels estimated show that across the EU as a whole there is a substantial margin for increasing research output without having to spend more resources. There are also major differences between countries in terms of inefficiency. The main sources of heterogeneity in scientific output in the HEIs of the EU are the differences in resources allocated per researcher and, to a lesser extent, the differences in efficiency within each knowledge field. The differences in quality and in specialisation also play a smaller role in determining differences in output.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Source: SCImago Journal & Country Rank, Eurostat and own elaboration

Fig. 2

Source: SCImago Journal & Country Rank, Eurostat and own elaboration

Fig. 3

Source: Eurostat

Fig. 4
Fig. 5

Source: SCImago Journal & Country Rank and Eurostat

Fig. 6

Source: SCImago Journal & Country Rank and own elaboration

Fig. 7

Source: SCImago Journal & Country Rank, Eurostat and own elaboration

Fig. 8

Source: SCImago Journal & Country Rank, Eurostat and own elaboration

Fig. 9
Fig. 10

Source: SCImago Journal & Country Rank, Eurostat and own elaboration

Fig. 11

Source: SCImago Journal & Country Rank, Eurostat and own elaboration

Fig. 12

Source: Own elaboration

Similar content being viewed by others

Notes

  1. Some studies propose the additional use of diverse indicators of the quality of university teaching, such as the drop-out rate, the performance rate, the student-teacher ratio, expenditure per student, the number of information technology (IT) and library staff per student, expenditure per student, etc. See Pérez et al. (2015a, b).

  2. Data provided by SCIMAGO Journal & Country Rank refer to the total number of scientific publications produces by a country. 99 % of the EU-28’s scientific output comes from universities (64.3 %), Public research centres (22.8 %) and Hospitals (11.8 %). For this reason the data on patents, publications, citations, R&D expenditure and R&D personnel provided throughout this paper refer to Higher Education (universities) and Government sector (Public Research Centres and Hospitals) as a whole.

  3. The information is available on the following website: http://www.scimagojr.com/countryrank.php

  4. R&D expenditures will cover also expenditures for R&D personnel. The data do not allow to disentangle personnel expenditure from other R&D expenditure by FOS. If we want to take into account the specialization effect, as we do, we have to use total R&D expenditures as input (which include both non-personnel R&D expenditure and research wages, in fact a measure of abundance of resources for researchers). It is also important to consider research as an output coming from more than one input. Labour is a very important input and other types of R&D expenditure are also very relevant. HEIs have the option of allocating more resources to their researchers, employing better qualified and paid researchers or using more researchers. We think that this is an important fact that needs to be considered. It is possible to disentangle personnel expenditure from other R&D expenditure but only without taking into account FOS specialization. We have carried out that exercise using two inputs: only the non-personnel R&D expenditure and R&D personnel. This analysis avoids any potential issue of double accounting. The coefficients of correlation between the inefficiency indicators obtained from this new exercise and the comparable inefficiency indicators obtained and discussed in the paper range between 0.96 and 0.98. Therefore, the results are maintained, showing robustness to this potential issue.

  5. The positive impacts of universities on the economic growth of their countries’ economies have been widely demonstrated in the literature, especially in the case of North American universities (Pastor et al. 2013).

  6. Alternatively, Farrel also proposed to measure efficiency from the perspective of the potential reduction of inputs given a vector of outputs (Farrell 1957).

  7. Maudos et al. (2000) use a similar methodology to analyse the regional output differences.

References

  • Abbott, M., & Doucouliagos, C. (2003). The efficiency of Australian universities: A data envelopment analysis. Economics of Education Review, 22(1), 89–97.

    Article  Google Scholar 

  • Abbott, M., & Doucouliagos, H. (2004). Research output of Australian universities. Education Economics, 12(3), 251–265.

    Article  Google Scholar 

  • Agrawal, A. K., & Henderson, R. M. (2002). Putting patents in context: Exploring knowledge transfer from MIT. Management Science, 48, 44–60.

    Article  Google Scholar 

  • Azoulay, P., Ding, W., & Stuart, T. (2009). The impact of academic patenting on the rate, quality and direction of (public) research output. The Journal of Industrial Economics, 57, 637–676.

    Article  Google Scholar 

  • Bonaccorsi, A., & Daraio, C. (2007). Universities and strategic knowledge creation: Specialization and performance in Europe. Cheltenham: Edward Elgar Publishing.

    Book  Google Scholar 

  • Breschi, L., Lissoni, F., & Montobbio, F. (2007). The scientific productivity of academic inventors: New evidence from Italian data. Economics of Innovation and New Technology, 16(2), 101–118.

    Article  Google Scholar 

  • Buenstorf, (2009). Is commercialization good or bad for science? Individual-level evidence from the Max Planck Society. Research Policy, 38(2), 281–292.

    Article  Google Scholar 

  • Carayol, N. (2007). Academic incentives, research organization and patenting at a large French University. Economics of Innovation and New Technology, 16(2), 119–138.

    Article  Google Scholar 

  • Center for Science and Technology Studies (CWTS). (2009). The Leiden ranking. Retrieved November from http://www.cwts.nl/ranking/LeidenRankingWebSite.html.

  • Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision-making units. European Journal of Operational Research, 2, 429–444. doi:10.1016/0377-2217(78)90138-8.

    Article  MathSciNet  MATH  Google Scholar 

  • Crespi, G., D’Este, P., Fontana, R., & Geuna, A. (2011). The impact of academic patenting on university research and its transfer. Research Policy, 40, 55–68.

    Article  Google Scholar 

  • de Groot, H., McMahon, W., & Volkwein, J. F. (1991). The cost structure of American Research Universities. The Review of Economics and Statistics, 73(3), 424–431.

    Article  Google Scholar 

  • Fabrizio, K., & DiMinin, A. (2008). Commercializing the laboratory: The relationship between faculty patenting and publishing. Research Policy, 37(5), 914–931.

    Article  Google Scholar 

  • Falagas, M. E., Kouranos, V. D., Arencibia-Jorge, R., & Karageorgopoulos, D. E. (2008). Comparison of SCImago journal rank indicator with journal impact factor. The FASEB Journal, 22(22), 2623–2628.

    Article  Google Scholar 

  • Farrell, M. (1957). The measurement of productive efficiency. Journal of the Royal Statistics Society, Series A, 120(3), 253–281.

    Article  Google Scholar 

  • Filippini, M., & Lepori, B. (2007). Cost structure, economies of capacity utilization and scope in Swiss higher education institutions. In Universities and strategic knowledge creation: Specialization and performance in Europe (pp. 272–304).

  • Flegg, A. T., Allen, D. O., Field, K., & Thurlow, T. W. (2004). Measuring the efficiency of British universities: A multi-period data envelopment analysis. Education Economics, 12(3), 231–249.

    Article  Google Scholar 

  • Glass, J. C., McKillop, D. G., & Hyndman, N. (1995). Efficiency in the provision of university teaching and research: An empirical analysis of UK universities. Journal of Applied Econometrics, 10(1), 61–72.

    Article  Google Scholar 

  • González-Albo, B., Moreno, L., Morillo, F., & Bordons, M. (2012). Bibliometric indicators for the analysis of the research performance of a multidisciplinary institution: The CSIC. Revista Española de Documentación Científica, 35(1), 9–37.

    Article  Google Scholar 

  • Guccio, C., Martorana, M. F., & Monaco, L. (2016). Evaluating the impact of the Bologna Process on the efficiency convergence of Italian universities: A non-parametric frontier approach. Journal of Productivity Analysis, 45(3), 275–298.

    Article  Google Scholar 

  • Johnes, G. (1988). Determinants of research output in economics departments in British universities. Research Policy, 17(3), 171–178.

    Article  Google Scholar 

  • Johnes, J. (1996). Performance assessment in higher education in Britain. European Journal of Operational Research, 89(1), 18–33.

    Article  MATH  Google Scholar 

  • Johnes, J. (2006). Data envelopment analysis and its application to the measurement of efficiency in higher education. Economics of Education Review, 25(3), 273–288.

    Article  MATH  Google Scholar 

  • Johnes, G., & Johnes, J. (2009). Higher education institutions’ costs and efficiency: Taking the decomposition a further step. Economics of Education Review, 28(1), 107–113. doi:10.1016/j.econedurev.2008.02.001.

    Article  Google Scholar 

  • Kempkes, G., & Pohl, C. (2010). The efficiency of German universities—some evidence from nonparametric and parametric methods. Applied Economics, 42(16), 2063–2079.

    Article  Google Scholar 

  • Klitkou, A., & Gulbrandsen, M. (2010). The relationship between academic patenting and scientific publishing in Norway. Scientometrics, 82(1), 93–108.

    Article  Google Scholar 

  • Kuah, C. T., & Wong, K. (2011). Efficiency assessment of universities through data envelopment analysis. Procedia Computer Science, 3, 499–506.

    Article  Google Scholar 

  • Lepori, B. (2007). Paterns in the Swiss higher education system. In Universities and strategic knowledge creation: Specialization and performance in Europe (pp. 209–240).

  • Maudos, J., Pastor, J. M., & Serrano, L. (2000). Efficiency and productive specialization: An application to the Spanish regions. Regional Studies, 34(9), 829–842.

    Article  Google Scholar 

  • Moed, H. F. (2005). Citation analysis in research evaluation. Dordrecht: Springer.

    Google Scholar 

  • Nazarko, J., & Šaparauskas, J. (2014). Application of DEA method in efficiency evaluation of public higher education institutions. Technological and Economic Development of Economy, 20(1), 25–44.

    Article  Google Scholar 

  • Pastor, J. M., Peraita, C., & Pérez, F. (2015a). Estimating the long-term economic impacts of Spanish universities on the National Economy. Papers in Regional Science,. doi:10.1111/pirs.12157.

    Google Scholar 

  • Pastor, J. M., Pérez, F., & Fernández de Guevara, J. (2013). Measuring the local economic impact of universities: An approach that considers uncertainty. Higher Education, 65(5), 539–564.

    Article  Google Scholar 

  • Pastor, J. M., Serrano, L., & Zaera, I. (2015b). The research output of European higher education institutions. Scientometrics, 102(3), 1867–1893.

    Article  Google Scholar 

  • Rey, O. (2009). Quality indicators and educational research publications: Which publications count?. Dossier d’actualité No. 46–June–July. Service de Veille scientifique et technologique, L’Institut Français de l’Éducation. http://ife.ens-lyon.fr/vst/DA/detailsDossier.php?parent=accueil&dossier=46&lang=en.

  • Salas, V. (2012). La producción universitaria. El caso chileno. Departamento de Economía. Universidad de Santiago de Chile.

  • Sarrico, C. S., & Dyson, R. G. (2004). Restricting virtual weights in data envelopment analysis. European Journal of Operational Research, 159(1), 17–34.

    Article  MathSciNet  MATH  Google Scholar 

  • Sarrico, C. S., Teixeira, P., Rosa, M. J., & Cardoso, M. F. (2009). Subject mix and productivity in Portuguese universities. European Journal of Operational Research, 197, 287–295.

    Article  MATH  Google Scholar 

  • Schubert, T., & Kroll, H. (2014). Universities’ effects on regional GDP and unemployment: The case of Germany. Papers in Regional Science,. doi:10.1111/pirs.12150.

    Google Scholar 

  • SCImago. (2012). SIR world report 2012. Global ranking. SCIMAGO Institutions Rankings (SIR). http://www.scimagoir.com.

  • SCImago. (2012). SJR—SCImago Journal & Country Rank. Retrieved July 2012, from http://www.scimagojr.com.

  • Stephan, P. E., Gurmu, S., Sumell, A. J., & Black, G. (2007). Who’s patenting in the university? Evidence from the survey of doctorate recipients. Economics of Innovation and New Technology, 16(2), 71–99.

    Article  Google Scholar 

  • Thanassoulis, E., Kortelainen, M., Johnes, G., & Johnes, J. (2011). Costs and efficiency of higher education institutions in England: A DEA analysis. Journal of the Operational Research Society, 62(7), 1282–1297. doi:10.1057/jors.2010.68.

    Article  Google Scholar 

  • Vieira, E. S., & Gomes, J. A. N. F. (2009). A comparison of scopus and web of science for a typical university. Scientometrics, 81(2), 587–600.

    Article  Google Scholar 

  • Worthington, A. C., & Lee, B. L. (2008). Efficiency, technology and productivity change in Australian universities, 1998–2003. Economics of Education Review, 27(3), 285–298.

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank a reviewer for the comments that helped to improve an earlier version of the manuscript. This paper was developed as part of the SPINTAN project funded by the European Commission. This project has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under Grant Agreement No: 612774. The authors are grateful for funding from Research Project ECO2015-70632-R (Ministry of the Economy and Competitiveness).

Author information

Authors and Affiliations

  1. Facultat d’Economia, Universitat de València, Av. de los Naranjos s/n, 46022, Valencia, Spain

    José Manuel Pastor

  2. Ivie, C/Guardia Civil 22, 46020, Valencia, Spain

    Lorenzo Serrano

Authors
  1. José Manuel Pastor
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lorenzo Serrano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to José Manuel Pastor.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pastor, J.M., Serrano, L. The determinants of the research output of universities: specialization, quality and inefficiencies. Scientometrics 109, 1255–1281 (2016). https://doi.org/10.1007/s11192-016-2102-3

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-016-2102-3

Keywords

JEL Classification

Search

Navigation