Abstract
This article analyzes the effect of country size, level of funding, method of financing, and ways of collaboration on scientific publication output in terms of the number of articles published and citations received in the scientific literature across national research systems. This article takes an initial step toward integrating mentioned aspects into one analysis because previously, they have been studied incoherently as separate issues. The study encompasses European countries using data provided by Clarivate Analytics, the European Commission and Eurostat. Based on the empirical analysis, three conclusions emerge. Firstly, we have to reject the proposition that the function of scientific production exhibits increasing returns to scale. Secondly, transnationally coordinated research projects have a strong positive effect on countries’ number of articles and citations. Smaller countries participate proportionally more in transnationally coordinated research and are therefore more affected by the phenomena of hyperauthorship. This explains why several small nations perform above their weight in impact relative to spending. Thirdly, the share of competitive project-based funding does not affect the number of articles published but has a U-shaped relationship with research impact per article, pointing toward two alternative financing strategies for maximizing impact based on high or low share of project-based funding. Based on the analysis, we present strengths and weaknesses of European countries’ research systems for policy purposes.
This is a preview of subscription content, log in via an institution to check access.
Source: author’s adaptation based on Lepori’s (2011) model
Source: author’s calculations based on Eurostat and Clarivate Analytics’ data (2008–2018)
Similar content being viewed by others
References
Abramo, G., Cicero, T., & D’Angelo, C. A. (2012). Revisiting size effects in higher education research productivity. Higher Education, 63(6), 701–717. https://doi.org/10.1007/s10734-011-9471-6
Abramo, G., D’Angelo, C. A., Di Costaa, F., & Solazzi, M. (2009). University–industry collaboration in Italy: A bibliometric examination. Technovation, 29(6–7), 498–507. https://doi.org/10.1016/j.technovation.2008.11.003
Abrigo, M. R., & Love, I. (2016). Estimation of panel vector autoregression in Stata. The Stata Journal, 16(3), 778–804. https://doi.org/10.1177/1536867X1601600314
Adams, J. (2012). The rise of research networks. Nature, 490(7420), 335–336. https://doi.org/10.1038/490335a
Adams, J. (2013). Collaborations: The fourth age of research. Nature, 497(7451), 557–560. https://doi.org/10.1038/497557a
Allik, J. (2013). Factors affecting bibliometric indicators of scientific quality. Trames: A Journal of the Humanities and Social Sciences, 17(3), 199–214. https://doi.org/10.3176/tr.2013.3.01
Allik, J., Lauk, K., & Realo, A. (2020). Factors predicting the scientific wealth of nations. Cross-Cultural Research. https://doi.org/10.1177/1069397120910982
Allison, P. D. (2009). Fixed effects regression models. SAGE Publications.
Arrow, K. (1962). The rate and direction of inventive activity: Economic and social factors. Princeton University Press.
Auranen, O., & Nieminen, M. (2010). University research funding and publication performance—An international comparison. Research Policy, 39(6), 822–834. https://doi.org/10.1016/j.respol.2010.03.003
Bloch, C., Ryan, T. K., & Andersen, J. P. (2019). Public-private collaboration and scientific impact: An analysis based on Danish publication data for 1995–2013. Journal of Informetrics, 13(2), 593–604. https://doi.org/10.1016/j.joi.2019.03.003
Bol, T., de Vaan, M., & van de Rijt, A. (2018). The Matthew effect in science funding. Proceedings of the National Academy of Sciences, 115(19), 4887–4890. https://doi.org/10.1073/pnas.1719557115
Bonaccorsi, A., & Daraio, C. (2005). Exploring size and agglomeration effects on public research productivity. Scientometrics, 63(1), 87–120. https://doi.org/10.1007/s11192-005-0205-3
Bornmann, L., & Haunschild, R. (2017). Does evaluative scientometrics lose its main focus on scientific quality by the new orientation towards societal impact? Scientometrics, 110(2), 937–943. https://doi.org/10.1007/s11192-016-2200-2
Castelvecchi, D. (2015). Physics paper sets record with more than 5000 authors. Nature. https://doi.org/10.1038/nature.2015.17567
Cimini, G., Gabrielli, A., & Labini, F. S. (2014). The scientific competitiveness of nations. PLoS ONE, 9(12), e113470. https://doi.org/10.1371/journal.pone.0113470
Clarivate Analytics. (2018). InCites indicators handbook. Clarivate Analytics.
Clermont, M., Dirksen, A., & Harald, D. (2015). Returns to scale of Business Administration research in Germany. Scientometrics, 103(2), 583–614. https://doi.org/10.1007/s11192-015-1561-2
Cocos, M., & Lepori, B. (2020). What we know about research policy mix. Science and Public Policy, 47(2), 1–11. https://doi.org/10.1093/scipol/scz061
Crespi, G. A., & Geuna, A. (2008). An empirical study of scientific production: A cross country analysis, 1981–2002. Research Policy, 37(4), 565–579. https://doi.org/10.1016/j.respol.2007.12.007
Cronin, B. (2001). Hyperauthorship: A postmodern perversion or evidence of a structural shift in scholarly communication practices? Journal of the American Society for Information Science and Technology, 52(7), 558–569. https://doi.org/10.1002/asi.1097
De Solla Price, D. J., & Beaver, D. (1966). Collaboration in an invisible college. American Psychologist, 21(11), 1011–1018. https://doi.org/10.1037/h0024051
Dialogic/Empirica. (2014). The effectiveness of national research funding systems. Utrecht/Bonn: Dialogic/Empirica. Retrieved from https://www.dialogic.nl/wp-content/uploads/2016/12/2013.109-1422.pdf.
Docampo, D., & Bessoule, J.-J. (2019). A new approach to the analysis and evaluation of the research output of countries and institutions. Scientometrics, 119(2), 1207–1225. https://doi.org/10.1007/s11192-019-03089-w
Eurostat. (2010). R&D budget statistics—transnationally coordinated research. Retrieved 19 Nov 2019, from https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Archive:R_%26_D_budget_statistics_-_transnationally_coordinated_research.
Eurostat. (2020). Government budget allocations for R&D (GBARD). Government budget allocations for R&D (GBARD). Retrieved from https://ec.europa.eu/eurostat/cache/metadata/en/gba_esms.htm#contact1616163588011.
Farshad, M., Claudia, S., & Gerber, C. (2013). Association of scientific and nonscientific factors to citation rates of articles of renowned orthopedic journals. European Orthopaedics and Traumatology, 4(3), 125–130. https://doi.org/10.1007/s12570-013-0174-6
Figg, W. D., Dunn, L., Liewehr, D., Steinberg, S. M., Thurman, P., Barrett, C., & Birkinshaw, J. (2006). Scientific collaboration results in higher citation. Pharmacotherapy, 26(6), 759–767. https://doi.org/10.1592/phco.26.6.759
Fisher, F. M., & Temin, P. (1973). Returns to scale in research and development: What does the Schumpeterian hypothesis imply? Journal of Political Economy, 81(1), 56–70. https://doi.org/10.1086/260006
Gantman, E. R. (2012). Economic, linguistic, and political factors in the scientific productivity of countries. Scientometrics, 93(3), 967–985. https://doi.org/10.1007/s11192-012-0736-3
García, C. E., & Sanz-Menéndez, L. (2005). Competition for funding as an indicator of research competitiveness. Scientometrics, 64(3), 271–300. https://doi.org/10.1007/s11192-005-0251-x
Garfield, E. (1979). Citation indexing– Its theory and application in science. John Wiley & Sons.
Gauffriau, M., & Larsen, P. O. (2005). Counting methods are decisive for rankings based on publication and citation studies. Scientometrics, 64(1), 85–93. https://doi.org/10.1007/s11192-005-0239-6
Gauffriau, M., Larsen, P. O., Maye, I., Roulin-Perriard, A., & von Ins, M. (2007). Publication, cooperation and productivity measures in scientific research. Scientometrics, 73(2), 175–214. https://doi.org/10.1007/s11192-007-1800-2
Georghiou, L. (2013). Effectiveness of national research systems. Discussion paper, Brussels. Retrieved from https://era.gv.at/object/document/366/attach/background_paper_session_1.pdf.
Glanzel, W. (2001). National characteristics in international scientific co-authorship relations. Scientometrics, 51(1), 69–115. https://doi.org/10.1023/A:1010512628145
Glänzel, W., & Schubert, A. (2005). Analysing scientific networks through co-authorship. In W. Glänzel & A. Schubert (Eds.), Handbook of quantitative science and technology research (pp. 257–276). Springer.
Goldfarb, A., & Tucker, C. (2019). Digital economics. Journal of Economic Literature, 57(1), 3–43. https://doi.org/10.1257/jel.20171452
Griliches, Z. (1979). Issues in assessing the contribution of research and development to productivity growth. The Bell Journal of Economics, 10(1), 92–116. https://doi.org/10.2307/3003321
Gross, K., & Bergstrom, C. T. (2019). Contest models highlight inherent inefficiencies of scientific funding competitions. PLOS Biology, 17(1), e3000065. https://doi.org/10.1371/journal.pbio.3000065
Guan, J., & Zuo, K. (2014). A cross-country comparison of innovation efficiency. Scientometrics, 100(2), 541–575. https://doi.org/10.1007/s11192-014-1288-5
Herron, P., Mehta, A., Cao, C., & Lenoir, T. (2016). Research diversification and impact: The case of national nanoscience development. Scientometrics, 109(2), 629–659. https://doi.org/10.1007/s11192-016-2062-7
Hirv, T. (2019). Research consortia determine a significant part of the bibliometric visibility of Estonian science. TRAMES: A Journal of the Humanities and Social Sciences, 23(3), 287–308. https://doi.org/10.3176/tr.2019.3.02
Huebner, J. (2005). A possible declining trend for worldwide innovation. Technological Forecasting and Social Change, 72(8), 980–986. https://doi.org/10.1016/j.techfore.2005.01.003
Iceland Monitor. (2015). Iceland has 10 of the world’s top scientists. Retrieved 5 June 2020, from https://www.mbl.is/ferdalog/frettir/2015/09/18/iceland_has_10_of_the_world_s_top_scientists/.
Jongbloed, B., & Lepori, B. (2015). Funding of research in higher education: Mixed models and mixed results. In The Palgrave international handbook of higher education policy and governance. Palgrave Macmillan. (pp. 439–462). https://doi.org/10.1007/978-1-137-45617-5_24.
Jonkers, K., & Zacharewicz, T. (2015). Performance based funding: A comparative assessment of their use and nature in EU Member States. Working paper. https://doi.org/10.2791/134058.
Jonkers, K., & Zacharewicz, T. (2016). Research performance based funding systems: A comparative assessment. Publications Office of the European Union. https://doi.org/10.2791/659483
King, C. (2012). Multiauthor papers: Onward and upward. Science Focus, 7, 62–64.
King, D. A. (2004). The scientific impact of nations. Nature, 430(6997), 311–316. https://doi.org/10.1038/430311a
Klein, S. G. (2015). Using performance-based funding to incentivize change. RTI International.
Koshal, R. K., & Koshal, M. (1999). Economies of scale and scope in higher education: A case of comprehensive universities. Economics of Education Review, 18(2), 269–277. https://doi.org/10.1016/S0272-7757(98)00035-1
Kulkarni, A. V., Busse, J. W., & Shams, I. (2007). Characteristics associated with citation rate of the medical literature. PLoS ONE, 2(5), e403. https://doi.org/10.1371/journal.pone.0000403
Laband, D. N., & Lentz, B. F. (2003). New estimates of economies of scale and scope in higher education. Southern Economic Journal, 70(1), 172–183. https://doi.org/10.2307/1061638
Lariviere, V., Gingras, Y., Sugimoto, C. R., & Tsou, A. (2015). Team size matters: Collaboration and scientific impact since 1900. Journal of the Association for Information Science and Technology, 66(7), 1323–1333. https://doi.org/10.1002/asi.23266
Laudel, G. (2006). The art of getting funded: How scientists adapt to their funding conditions. Science and Public Policy, 33(7), 489–504. https://doi.org/10.3152/147154306781778777
Lauk, K., & Allik, J. (2018). A puzzle of Estonian science: How to explain unexpected rise of the scientific impact. Trames: A Journal of the Humanities and Social Sciences, 22(4), 329–344. https://doi.org/10.3176/tr.2018.4.01
Lebeau, L.-M., Laframboise, M.-C., Larivière, V., & Gingras, Y. (2008). The effect of university–industry collaboration on the scientific impact of publications: The Canadian case, 1980–2005. Research Evaluation, 17(3), 227–232. https://doi.org/10.3152/095820208X331685
Lepori, B. (2011). Coordination modes in public funding systems. Research Policy, 40(3), 355–367. https://doi.org/10.1016/j.respol.2010.10.016
Lepori, B., Masso, J., Jabłecka, J., Sima, K., & Ukrainski, K. (2009). Comparing the organization of public research funding in central and eastern European countries. Science and Public Policy, 36(9), 667–681. https://doi.org/10.3152/030234209X479494
Lepori, B., van den Besselaar, P., Dinges, M., van der Meulen, B., Potì, B., Reale, E., Slipsersaeter, S., & Theves, J. (2007). Indicators for comparative analysis of public project funding: Concepts, implementation and evaluation. Research Evaluation, 16(4), 243–255. https://doi.org/10.3152/095820207X260252
Leydesdorff, L., & Henry, E. (1996). Emergence of a Triple Helix of university—industry—government relations. Science and Public Policy, 23(5), 279–286. https://doi.org/10.1093/spp/23.5.279
Leydesdorff, L., & Wagner, C. (2009). Macro-level indicators of the relations between research funding and research output. Journal of Informetrics, 3(4), 353–362. https://doi.org/10.1016/j.joi.2009.05.005
Leydesdorff, L., Bornmann, L., & Wagner, C. S. (2018). The relative influences of government funding and international collaboration on citation impact. Journal of the Association for Information Science and Technology, 70(2), 198–201. https://doi.org/10.1002/asi.24109
May, R. M. (1997). The scientific wealth of nations. Science, 275(5301), 793–796. https://doi.org/10.1126/science.275.5301.793
Moed, H. F. (2005). Citation analysis in research evaluation. Springer.
Morillo, F. (2016). Public-private interactions reflected through the funding acknowledgements. Scientometrics, 108(3), 1193–1204. https://doi.org/10.1007/s11192-016-2032-0
Mueller, C. (2016). Accurate forecast of countries’ research output by macro-level indicators. Scientometrics, 109(2), 1307–1328. https://doi.org/10.1007/s11192-016-2084-1
Must, Ü. (2014). The impact of multi-authored papers: The case of a small country. COLLNET Journal of Scientometrics and Information Management, 8(1), 41–47. https://doi.org/10.1080/09737766.2014.916874
Nelson, R. R. (1959). The simple economics of basic scientific research. Journal of Political Economy, 67(3), 297–306. https://doi.org/10.1086/258177
OECD. (2006). Knowledge-based economy. Retrieved 11 Feb 2020, from https://stats.oecd.org/glossary/detail.asp?ID=6864.
OECD. (2018). Effective operation of competitive research funding systems. OECD Publishing.
Pacchioni, G. (2018). The overproduction of truth: Passion, competition, and integrity in modern science. Oxford University Press.
Parker, J. N., Allesina, S., & Lortie, C. J. (2013). Characterizing a scientific elite (B): Publication and citation patterns of the most highly cited scientists in environmental science and ecology. Scientometrics, 94, 469–480. https://doi.org/10.1007/s11192-012-0859-6
Pastor, J. M., & Serrano, L. (2016). The determinants of the research output of universities: Specialization, quality and inefficiencies. Scientometrics, 109(2), 1255–1281. https://doi.org/10.1007/s11192-016-2102-3
Powell, J. J., & Dusdal, J. (2017). Science production in Germany, France, Belgium, and Luxembourg: Comparing the contributions of research Universities and Institutes to science, technology, engineering, mathematics, and health. Minerva, 55(4), 413–434. https://doi.org/10.1007/s11024-017-9327-z
Prathap, G. (2017). Scientific wealth and inequality within nations. Scientometrics, 113(2), 923–928. https://doi.org/10.1007/s11192-017-2511-y
Puuska, H.-M., Muhonen, R., & Leino, Y. (2013). International and domestic co-publishing and their citation impact in different disciplines. Scientometrics, 98(2), 823–839. https://doi.org/10.1007/s11192-013-1181-7
Raudla, R., Karo, E., Valdmaa, K., & Kattel, R. (2015). Implications of project-based funding of research on budgeting and financial management in public universities. Higher Education, 70(6), 957–971. https://doi.org/10.1007/s10734-015-9875-9
Reale, E. (2017). Analysis of national public research funding-PREF. Final Report. https://doi.org/10.2760/19140
Reale, E., Lepori, B., Nedeva, M., Thomas, D., Primeri, E., Chassagneux, E., & Laredo, P. (2013). Investments in joint and open research programmes and analysis of their economic impact (JOREP). Directorate General for Research and Innovation.
Romer, D. (2012). Advanced macroeconomics (Ed.). McGraw-Hill Education.
Rousseau, S., & Rousseau, R. (1997). The scientific wealth of European nations: Taking effectiveness into account. Scientometrics, 42(1), 75–87. https://doi.org/10.1007/BF02465013
Salomon-Fernandez, Y. (2014). The Massachusetts community college performance-based funding formula: A new model for New England?. New England Journal of Higher Education.
Sandstrom, U., & Van den Besselaar, P. (2018). Funding, evaluation, and the performance of national research systems. Journal of Informetrics, 12(1), 365–384. https://doi.org/10.1016/j.joi.2018.01.007
Schumpeter, J. (1942). Capitalism, socialism and democracy. Harper & Brothers.
Sharma, S., & Thomas, V. (2008). Inter-country R&D efficiency analysis: An application of data envelopment analysis. Scientometrics, 76(3), 483–501. https://doi.org/10.1007/s11192-007-1896-4
Shelton, R. D. (2020). Scientometric laws connecting publication counts to national research funding. Scientometrics, 123(1), 181–206. https://doi.org/10.1007/s11192-020-03392-x
Sugimoto, C. R., & Larivière, V. (2018). Measuring research: What everyone needs to know. Oxford University Press.
Surry, D. U. (2008). Innovations in Learning Technology. In L. A. Tomei, Encyclopedia of information technology curriculum integration. Hershey: IGI Global. (pp. 413–417). https://doi.org/10.4018/978-1-59904-881-9
Thelwall, M. (2020). Large publishing consortia produce higher citation impact research but coauthor contributions are hard to evaluate. Quantitative Science Studies, 1, 290–302. https://doi.org/10.1162/qss_a_00003
Todeschini, R., & Baccini, A. (2016). Handbook of bibliometric indicators: Quantitative tools for studying and evaluating research. John Wiley & Sons.
Trachana, V. (2013). Austerity-led brain drain is killing Greek science. Nature, 496(7445), 271–271. https://doi.org/10.1038/496271a
Van Steen, J. (2012). Modes of public funding of research and development: Towards internationally comparable indicators. OECD Publishing.
Van Leeuwen, T. N., Moed, H. F., Tijssen, R. J., Visser, M. S., & Van Raan, A. F. (2001). Language biases in the coverage of the Science Citation Index and its consequencesfor international comparisons of national research performance. Scientometrics, 51(1), 335–346. https://doi.org/10.1023/A:1010549719484
Vinkler, P. (2008). Correlation between the structure of scientific research, scientometric indicators and GDP in EU and non-EU countries. Scientometrics, 74(2), 237–254. https://doi.org/10.1007/s11192-008-0215-z
Vinkler, P. (2018). Structure of the scientific research and science policy. Scientometrics, 114(2), 737–756. https://doi.org/10.1007/s11192-017-2568-7
Wagner, C. S., & Jonkers, K. (2017). Open countries have strong science. Nature, 550(7674), 32–33. https://doi.org/10.1038/550032a
Wagner, C. S., Whetsell, T., Baas, J., & Jonkers, K. (2018). Openness and impact of leading scientific countries. Frontiers in Research Metrics and Analytics. https://doi.org/10.3389/frma.2018.00010
Waltman, L. (2016). A review of the literature on citation impact indicators. Journal of Informetrics, 10(2), 365–391. https://doi.org/10.1016/j.joi.2016.02.007
Woodley of Menie, M. A., Figueredo, A. J., & Sarraf, M. A. (2019). Slowing life history (K) can account for increasing micro-innovation rates and GDP growth, but not macro-innovation rates, which declined following the end of the Industrial Revolution. The Behavioral and brain sciences. https://doi.org/10.1017/S0140525X19000098.
Worthington, A. C., & Higgs, H. (2011). Economies of scale and scope in Australian higher education. Higher Education, 61(4), 387–414. https://doi.org/10.1007/s10734-010-9337-3
Zacharewicz, T., Lepori, B., Reale, E., & Jonkers, K. (2019). Performance-based research funding in EU Member States—a comparative assessment. Science and Public Policy, 46, 105–115.
Zanotto, S. R., Haeffner, C., & Guimaraes, J. A. (2016). Unbalanced international collaboration affects adversely the usefulness of countries’ scientific output as well as their technological and social impact. Scientometrics, 109, 1789–1814.
Zubașcu, F. (2020). Call to reform Slovakia’s R&D system. Science|Business. Retrieved 10 Apr 2020 from https://sciencebusiness.net/news/call-reform-slovakias-rd-system.
Acknowledgements
The author thanks Bradley Loewen for proofreading and anonymous reviewers for helpful comments.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hirv, T. The interplay of the size of the research system, ways of collaboration, level, and method of funding in determining bibliometric outputs. Scientometrics 127, 1295–1316 (2022). https://doi.org/10.1007/s11192-021-04232-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-021-04232-2