Skip to main content
SpringerLink
Log in

What is the relationship between research funding and citation-based performance? A comparative analysis between critical disciplines

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

The goal of the study here is to model and analyze the relation between research funding and citation-based performance in science to predict the diffusion of new scientific results in society. In fact, an important problem in the field of scientometrics is to explain factors determining the growth of citations in documents that can increase the diffusion of scientific results and the impact of science on society. The study here confronts this problem by developing a scientometric analysis to clarify, whenever possible, the relation between research funding and citations of articles in critical disciplines. Data of 2015 retrieved from the Web of Science database relating to the three critical disciplines given by computer science, medicine and economics are analyzed. Results suggest that computer science journals published more funded than unfunded papers. Medicine journals published equally funded and unfunded documents, and finally economics journals published more unfunded than funded papers. In addition, funded documents received more citations than unfunded papers in all three disciplines under study. The study also finds that citations in funded, unfunded and total (funded + unfunded) papers follow a power-law distribution in different disciplines. Another novel finding is that for all disciplines under study, the Matthew effect is greater for funded articles compared to unfunded documents. The results here can support best practices of research policy directed to fund vital scientific research for increasing the diffusion of science and scientific findings in society.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig.1

Similar content being viewed by others

References

  • Alstott, J., Bullmore, E., & Plenz, D. (2014). Powerlaw: A Python package for analysis of heavy-tailed distributions. PLoS ONE, 9(1), e85777. https://doi.org/10.1371/journal.pone.0085777

    Article  Google Scholar 

  • Barabási, A. L., & Albert, R. (1999). Emergence of scaling in random networks. Science, 286(5439), 509–512.

    Article  MathSciNet  Google Scholar 

  • Clauset, A., Shalizi, C. R., & Newman, M. E. J. (2009). Power-law distributions in empirical data. SIAM Review, 51(4), 661–703.

    Article  MathSciNet  Google Scholar 

  • Coccia, M. (2005a). A Scientometric model for the assessment of scientific research performance within public institutes. Scientometrics, 65(3), 307–321. https://doi.org/10.1007/s11192-005-0276-1

    Article  Google Scholar 

  • Coccia, M. (2005b). A taxonomy of public research bodies: A systemic approach. Prometheus, 23(1), 63–82. https://doi.org/10.1080/0810902042000331322

    Article  Google Scholar 

  • Coccia, M. (2018). General properties of the evolution of research fields: A scientometric study of human microbiome, evolutionary robotics and astrobiology. Scientometrics, 117(2), 1265–1283. https://doi.org/10.1007/s11192-018-2902-8

    Article  Google Scholar 

  • Coccia, M. (2019). Why do nations produce science advances and new technology? Technology in society, 59, 101124. https://doi.org/10.1016/j.techsoc.2019.03.007

    Article  Google Scholar 

  • Coccia, M. (2020). The evolution of scientific disciplines in applied sciences: Dynamics and empirical properties of experimental physics. Scientometrics, 124, 451–487. https://doi.org/10.1007/s11192-020-03464-y

    Article  Google Scholar 

  • Coccia, M., & Bozeman, B. (2016). Allometric models to measure and analyze the evolution of international research collaboration. Scientometrics, 108(3), 1065–1084. https://doi.org/10.1007/s11192-016-2027-x

    Article  Google Scholar 

  • Coccia, M., & Wang, L. (2016). Evolution and convergence of the patterns of international scientific collaboration. Proceedings of the National Academy of Sciences of the United States of America, 113(8), 2057–2061. https://doi.org/10.1073/pnas.1510820113

    Article  Google Scholar 

  • Coccia, M., Falavigna, G., & Manello, A. (2015). The impact of hybrid public and market-oriented financing mechanisms on scientific portfolio and performances of public research labs: A scientometric analysis. Scientometrics, 102(1), 151–168. https://doi.org/10.1007/s11192-014-1427-z

    Article  Google Scholar 

  • Cronin, B., & Shaw, D. (1999). Citation, funding acknowledgment and author nationality relationships in four information science journals. Journal of Documentation, 55(4), 402–408.

    Article  Google Scholar 

  • de Solla Price, D. J. (1976). A general theory of bibliometric and other cumulative advantage processes. Journal of the American Society for Information Science, 27, 292–306.

    Article  Google Scholar 

  • Fortunato, S., Bergstrom, C. T., Börner, K., Evans, J. A., Helbing, D., Milojević, S., et al. (2018). Science of science. Science. https://doi.org/10.1126/science.aao0185

    Article  Google Scholar 

  • Gao, J. P., Su, C., Wang, H. Y., Zhai, L. H., & Pan, Y. T. (2019). Research fund evaluation based on academic publication output analysis: The case of Chinese research fund evaluation. Scientometrics, 119(2), 959–972.

    Article  Google Scholar 

  • Gillespie, C. S. (2015). Fitting heavy tailed distributions: The poweRlawpackage. Journal of Statistical Software, 64, 1–16.

    Article  Google Scholar 

  • Gök, A., Rigby, J., & Shapira, P. (2016). The impact of research funding on scientific outputs: Evidence from six smaller European countries. Journal of the Association for Information Science and Technology, 67(3), 715–730. https://doi.org/10.1002/asi.23406

    Article  Google Scholar 

  • Hicks, D., & Katz, J. S. (2011). Equity and excellence in research funding. Minerva, 49(2), 137–151.

    Article  Google Scholar 

  • Huang, Z., Chen, H., Li, X., & Roco, M. C. (2006). Connecting NSF funding to patent innovation in nanotechnology (2001–2004). Journal of Nanoparticle Research, 8(6), 859–879.

    Article  Google Scholar 

  • Jacob, B. A., & Lefgren, L. (2011). The impact of research grant funding on scientific productivity. Journal of Public Economics, 95(9–10), 1168–1177.

    Article  Google Scholar 

  • Katz, J. S. (2000). Scale-independent indicators and research evaluation. Science and Public Policy, 27(1), 23–36.

    Article  Google Scholar 

  • Katz, J. S. (2005). Scale-independent bibliometric indicators. Measurement: Interdisciplinary Research and Perspectives., 3(1), 24–28. https://doi.org/10.1207/s15366359mea0301_3

    Article  Google Scholar 

  • Katz, J. S. (2016). What is a complex innovation system? PLoS ONE, 11(6), e0156150.

    Article  Google Scholar 

  • Leguendre, P., & Leguendre, L. (2012). Numerical ecology (3rd ed.). Elsevier B. V.

    Google Scholar 

  • Lewison, G., & Dawson, G. (1998). The effect of funding on the outputs of biomedical research. Scientometrics, 41, 17–27. https://doi.org/10.1007/BF02457963

    Article  Google Scholar 

  • Lotka, A. J. (1926). The frequency distribution of scientific productivity. Journal of the Academy of Sciences, 16(1), 317–323.

    Google Scholar 

  • Merton, R. K. (1988). The Matthew effect in science, II: Cumulative advantage and the symbolism of intellectual property. Isis, 79(4), 606–623.

    Article  Google Scholar 

  • Morillo, F. (2020). Is open access publication useful for all research fields? Presence of funding, collaboration and impact. Scientometrics, 125, 689–716. https://doi.org/10.1007/s11192-020-03652-w

    Article  Google Scholar 

  • Pao, M. L. (1991). On the relationship of funding and research publications. Scientometrics, 20, 257–281. https://doi.org/10.1007/BF02018158

    Article  Google Scholar 

  • Quinlan, K. M., Kane, M., & Trochim, W. M. K. (2008). Evaluation of large research initiatives: Outcomes, challenges, and methodological considerations. New Directions for Evaluation, 118, 61–72. https://doi.org/10.1002/ev.261

    Article  Google Scholar 

  • Rigby, J. (2013). Looking for the impact of peer review: Does count of funding acknowledgements really predict research impact? Scientometrics, 94(1), 57–73.

    Article  Google Scholar 

  • Ronda-Pupo, G. A. (2017). The citation-based impact of complex innovation systems scales with the size of the system. Scientometrics, 112(1), 141–151.

    Article  Google Scholar 

  • Ronda-Pupo, G. A., & Katz, J. S. (2016a). The scaling relationship between citation-based performance and coauthorship patterns in natural sciences. Journal of the Association for Information Science and Technology, 68(5), 1257–1265.

    Article  Google Scholar 

  • Ronda-Pupo, G. A., & Katz, J. S. (2016b). The power–law relationship between citation-based performance and collaboration in articles in management journals: A scale-independent approach. Journal of the Association for Information Science and Technology, 67(10), 2565–2572.

    Article  Google Scholar 

  • Ronda-Pupo, G. A., & Katz, J. S. (2018). The power law relationship between citation impact and multi-authorship patterns in articles in Information Science & Library Science journals. Scientometrics, 114(3), 919–932.

    Article  Google Scholar 

  • Rousseau, R (2000, September). Are multi-authored articles cited more than single-authored ones? Are collaborations with authors from other countries more cited than collaborations whitin the country? A case study. In Proceedings of the second Berlin workshop on scientometrcs and informetrics. Collaboration in Sceince and Technology. Gesellschaft furr Wissenschaftsforschung: Berlin (pp.173-176)

  • Shen, C.-C., Hu, Y.-H., Lin, W.-C., Tsai, C.-F., & Ke, S.-W. (2016). Research impact of general and funded papers. Online Information Review, 40(4), 472–480. https://doi.org/10.1108/OIR-08-2015-0249

    Article  Google Scholar 

  • Su, A. I., & Hogenesch, J. B. (2007). Power-law-like distributions in biomedical publications and research funding. Genome Biology, 8(4), 404. https://doi.org/10.1186/gb-2007-8-4-404

    Article  Google Scholar 

  • Van Raan, A. (1998). The influence of international collaboration on the impact of research results: Some simple mathematical considerations concerning the role of self-citations. Scientometrics, 42(3), 423–428.

    Article  Google Scholar 

  • Van Raan, A. F. J. (2006). Statistical properties of bibliometric indicators: Research group indicator distributions and correlations. Journal of the American Society for Information Science and Technology, 57(3), 408–430. https://doi.org/10.1002/asi.20284

    Article  Google Scholar 

  • Van Raan, A. F. V. (2008). Bibliometric statistical properties of the 100 largest European research universities: Prevalent scaling rules in the science system. Journal of the American Society for Information Science and Technology, 59(3), 461–475.

    Article  Google Scholar 

  • Wang, J., & Shapira, P. (2011). Funding acknowledgement analysis: An enhanced tool to investigate research sponsorship impacts: The case of nanotechnology. Scientometrics, 87(3), 563–586.

    Article  Google Scholar 

  • Wang, J., & Shapira, P. (2015). Is there a relationship between research sponsorship and publication impact? An analysis of funding acknowledgments in nanotechnology papers. PloS ONE, 10(2), e0117727.

    Article  Google Scholar 

  • Web of Science. (2021). Web of Science, Search in: Web of Science Core Collection, http://apps.webofknowledge.com/WOS_GeneralSearch_input.do?product=WOS&SID=E5lSYgaLwJn6kp2iz2G&search_mode=GeneralSearch (Accessed 24 March 2021)

  • Yan, E., Wu, C., & Song, M. (2018). The funding factor: A cross-disciplinary examination of the association between research funding and citation impact. Scientometrics, 115(1), 369–384.

    Article  Google Scholar 

  • Ye, F. Y., & Rousseau, R. (2008). The power law model and total career h-index sequences. Journal of Informetrics, 2(4), 288–297. https://doi.org/10.1016/j.joi.2008.09.002

    Article  Google Scholar 

  • Zhao, S. X., Lou, W., Tan, A. M., & Yu, S. (2018). Do funded papers attract more usage? Scientometrics, 115(1), 153–168.

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank Professor J. Sylvan Katz for his helpful instruction on power-law analysis.

Author information

Authors and Affiliations

  1. Department of Industrial Management, Faculty of Management and Accounting, Allameh Tabataba’i University, Tehran, Iran

    Saeed Roshani & Mohammad-Reza Bagherylooieh

  2. Birmingham Business School, College of Social Sciences, University of Birmingham, Birmingham, UK

    Melika Mosleh

  3. CNR—National Research Council of Italy Collegio Carlo Alberto, Via Real Collegio, 30-10024, Moncalieri, Torino, Italy

    Mario Coccia

Authors
  1. Saeed Roshani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad-Reza Bagherylooieh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Melika Mosleh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mario Coccia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Saeed Roshani.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Roshani, S., Bagherylooieh, MR., Mosleh, M. et al. What is the relationship between research funding and citation-based performance? A comparative analysis between critical disciplines. Scientometrics 126, 7859–7874 (2021). https://doi.org/10.1007/s11192-021-04077-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-021-04077-9

Keywords

Search

Navigation