Skip to main content
SpringerLink
Log in

Is there a correlation between journal impact factor and researchers’ performance? A study comprising the fields of clinical nephrology and neurosciences

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Quantifying the scientific performance of investigators has become an integral part of decision-making in research policy. The aim of the present study was to evaluate if there is a correlation between journal impact factor (IF) and researchers’ influence among a selected group of Brazilian investigators in the fields of clinical nephrology and neurosciences. This study was based on 94 senior investigators (36 in clinical nephrology and 58 in clinical neurosciences) receiving productivity scholarships from the Brazilian Council for Scientific and Technological Development (CNPq) according to a list provided by the agency in February 2009. Scientific performance indicators included in the analysis were: number of papers indexed by the Web of Science and Scopus databases, number of citations, h- and m-index. IFs were analyzed as (1) cumulative IF (∑IF), (2) IF adjusted by time (IF/t), and (3) average IF. There was a moderate positive correlation only between ∑IF and two indicators: total number of citations (P < 0.001) and h-index (P < 0.001). There was also a positive correlation between IF/t and m-index (P < 0.001). There was an agreement in these correlations between both groups (clinical nephrology and neurosciences). No significant correlation between the average IF and any of the scientific indicators was detected. A cut-off of 10.53 for IF/t showed the best performance in predicting researchers with m-index equal to or greater than 1. According to our findings, other qualitative and quantitative instruments rather than IF are clearly needed for identifying researchers with outstanding scientific output.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Abramo, G., D’Angelo, C. A., & Costa, F. (2011). National research assessment exercises: The effects of changing the rules of the game during the game. Scientometrics, 88, 229–238.

    Article  Google Scholar 

  • Abt, H. A. (2011). A publication index that is independent of age. Scientometrics,. doi:10.1007/s11192-011-0525-4.

    Google Scholar 

  • Balaban, A. T. (2012). Positive and negative aspects of citation indices and journal impact factors. Scientometrics, 92, 241–247.

    Article  Google Scholar 

  • Benitez-Bribiesca, L. (1999). The impact factor of medical journals: Its use and misuse. Archives of Medical Research, 30(3), 161–162.

    Article  Google Scholar 

  • Boell, S. K., & Wilson, C. S. (2010). Journal Impact Factors for evaluating scientific performance: Use of h-like indicators. Scientometrics, 82, 613–626.

    Article  Google Scholar 

  • Bordons, M., Fernandez, M. T., & Gomes, I. (2002). Advantages and limitations in the use of impact factor measures for the assessment of research performance in a peripheral country. Scientometrics, 53(2), 195–206.

    Article  Google Scholar 

  • Bornmann, L., & Daniel, H.-D. (2008). What do citation counts measure? A review of studies on citing behavior. Journal of Documentation, 64(1), 45–80.

    Article  Google Scholar 

  • Bornmann, L., Mutz, R., & Daniel, H.-D. (2008a). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology, 59(5), 830–837.

    Article  Google Scholar 

  • Bornmann, L., Mutz, R., Neuhaus, C., & Daniel, H.-D. (2008b). Citation counts for research evaluation: Standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics in Science and Environmental Politics, 8, 93–102.

    Article  Google Scholar 

  • Bornmann, L., Wallon, G., & Ledin, A. (2008c). Is the h index related to (standard) bibliometric measures and to the assessments by peers? An investigation of the h index by using molecular life sciences data. Research Evaluation, 17(2), 149–156.

    Article  Google Scholar 

  • Braun, T. (2012). Editorial. Scientometrics, 92, 207–208.

    Article  Google Scholar 

  • Brink, A. J. (2004). Impact factor: Use and abuse. Cardiovascular Journal of South Africa, 15(1), 5–7.

    Google Scholar 

  • Brown, H. (2007). How impact factors changed medical publishing—and science. BMJ, 334(7593), 561–564. doi:10.1136/bmj.39142.454086.AD.

    Article  Google Scholar 

  • Cardoso, S. C., & Gattas, G. J. (2009). The scientific production of full professors of the Faculdade de Medicina da Universidade de Sao Paulo: A view of the period of 2001–2006. Clinics (Sao Paulo), 64(9), 903–909. doi:34910.1590/S1807-59322009000900012.

    Article  Google Scholar 

  • CNPq, Ministério de Ciência e Tecnologia (2012). Critérios de Julgamento—CA-MD. Retrieved September 4, 2012 from http://www.cnpq.br/cas/ca-md.htm#criterios.

  • Correa, A. A., Moreira-Almeida, A., Meneze, P. R., Vallada, H., & Scazufca, M. (2011). Investigating the role played by social support in the association between religiosity and mental health in low income older adults: Results from the Sao Paulo Ageing and Health Study (SPAH). Revista Brasileira de Psiquiatria, 33(2), 157–164.

    Google Scholar 

  • Dong, P., Loh, M., & Mondry, A. (2005). The “impact factor” revisited. Biomedical Digital Libraries, 2, 7. doi:10.1186/1742-5581-2-7.

    Article  Google Scholar 

  • Editorial. (2012). Count on me. Nature, 489, 177.

    Google Scholar 

  • Franceschini, F., & Maisano, D. (2011). Proposals for evaluating the regularity of a scientist’s research output. Scientometrics, 88, 279–295.

    Article  Google Scholar 

  • Grzybowski, A. (2009). The journal impact factor: How to interpret its true value and importance. Medical Science Monitor, 15(2), SR1–SR4.

    Google Scholar 

  • Grzybowski, A. (2010). Impact factor—strengths and weaknesses. Clinics in Dermatology, 28(4), 455–457. doi:10.1016/j.clindermatol.2010.01.002.

    Article  MathSciNet  Google Scholar 

  • Haeffner-Cavaillon, N., & Graillot-Gak, C. (2009). The use of bibliometric indicators to help peer-review assessment. Archivum Immunolgiae et Therapiae Experimentalis, 57(1), 33–38. doi:10.1007/s00005-009-0004-2.

    Article  Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of United States of America, 102(46), 16569–16572. doi:10.1073/pnas.0507655102.

    Article  Google Scholar 

  • Jeang, K. T. (2008). H-index, mentoring-index, highly-cited and highly-accessed: How to evaluate scientists? Retrovirology, 5, 106. doi:10.1186/1742-4690-5-106.

    Article  Google Scholar 

  • Kelly, C. D., & Jennions, M. D. (2006). The h index and career assessment by numbers. Trends in Ecology and Evolution, 21(4), 167–170. doi:10.1016/j.tree.2006.01.005.

    Article  Google Scholar 

  • Kumar, V., Upadhyay, S., & Medhi, B. (2009). Impact of the impact factor in biomedical research: Its use and misuse. Singapore Medical Journal, 50(8), 752–755.

    Google Scholar 

  • Lehmann, S., Jackson, A. D., & Lautrup, B. E. (2006). Measures for measures. Nature, 444(7122), 1003–1004. doi:10.1038/4441003a.

    Article  Google Scholar 

  • Martelli-Júnior, H., Martelli, D. R., Quirino, I. G., Oliveira, M. C. L., Lima, L. S., & Oliveira, E. A. (2010). CNPq researchers in medicine: A comparative study of research areas. Revista da Associação Médica Brasile, 56(4), 478–483.

    Article  Google Scholar 

  • Metze, K. (2010). Bureaucrats, researchers, editors, and the impact factor: A vicious circle that is detrimental to science. Clinics (Sao Paulo), 65(10), 937–940.

    Article  Google Scholar 

  • Oliveira, E. A., Colosimo, E. A., Martelli, D. R., Quirino, I. G., Oliveira, M. C. L., Silva, L. S., et al. (2011a). Comparison of Brazilian researchers in clinical medicine: Are criteria for ranking well-adjusted? Scientometrics, 90, 429–443. doi:10.1007/s11192-011-0492-9.

    Article  Google Scholar 

  • Oliveira, E. A., Peicots-Filho, R., Quirino, I. G., Oliveira, M. C. L., Martelli, D. R., Lima, L. S., et al. (2011b). Perfil e produção científica dos pesquisadores do CNPq nas áreas de Nefrologia e Urologia. Jornal Brasileiro de Nefrologia, 33, 31–37.

    Article  Google Scholar 

  • Oliveira, E. A., Ribeiro, A. L. P., Quirino, I. G., Oliveira, M. C. L., Martelli, D. R., Lima, L. S., et al. (2011c). Perfil e produção científica dos pesquisadores do Conselho Nacional de Desenvolvimento Científico e Tecnológico na área de Cardiologia. Arquivos Brasileiro de Cardiologia, 97(3), 186–193.

    Article  Google Scholar 

  • Opthof, T. (1997). Sense and nonsense about the impact factor. Cardiovascular Research, 33(1), 1–7.

    Article  Google Scholar 

  • Panaretos, J., & Malesios, C. (2009). Assessing scientific research performance and impact with single indices. Scientometrics, 81(3), 635–670.

    Article  Google Scholar 

  • Rothenberg, R. (2008). The impact factor follies. Epidemiology, 19(3), 372. doi:10.1097/EDE.0b013e31816b6a8c.

    Article  Google Scholar 

  • Santos, N. C. F., Candido, L. F. O., & Kuppens, C. L. (2010). Produtividade em pesquisa do CNPq: análise do perfil dos pesquisadores da química. Quimica Nova, 33(2), 489–495.

    Article  Google Scholar 

  • Seglen, P. O. (1994). Causal relationship between article citedness and journal impact. Journal of the American Society for Information Science, 45(1), 1–11.

    Article  Google Scholar 

  • Seglen, P. O. (1997). Why the impact factor of journals should not be used for evaluating research. BMJ, 314(7079), 498–502.

    Article  Google Scholar 

  • Shibayama, S. (2011). Distribution of academic research funds: A case of Japanese national research grant. Scientometrics, 88, 43–60.

    Article  Google Scholar 

  • Simons, K. (2008). The misused impact factor. Science, 322(5899), 165. doi:10.1126/science.1165316.

    Article  Google Scholar 

  • Taylor, M., Perakakis, P., & Trachana, V. (2008). The siege of science. Ethics in Science and Environmental Politics, 8, 17–40.

    Article  Google Scholar 

  • Tess, B. H., Furuie, S. S., Castro, R. C., Barreto Mdo, C., & Nobre, M. R. (2009). Assessing the scientific research productivity of a Brazilian healthcare institution: A case study at the Heart Institute of Sao Paulo, Brazil. Clinics (Sao Paulo), 64(6), 571–576.

    Article  Google Scholar 

  • van Leeuwen, T. (2012). Discussing some basic critique on Journal Impact Factors: Revision of earlier comments. Scientometrics, 92, 443–455.

    Article  Google Scholar 

  • Vinkler, P. (1986). Evaluation of some methods for the relative assessment of scientific publications. Scientometrics, 10, 157–177.

    Article  Google Scholar 

  • Zitt, M. (2012). The journal impact factor: Angel, devil, or scapegoat? A comment on J. K. Vanclay’s article 2011. Scientometrics, 92, 485–503.

    Article  Google Scholar 

Download references

Acknowledgments

This study was partially supported by CNPq (National Council for Scientific and Technological Development) and FAPEMIG (Research Support Foundation of Minas Gerais). Eduardo A. Oliveira, Ana Cristina Simões e Silva, and Roberto Peicots-Filho were the recipients of CNPq scholarships in the area of medicine and their main area of investigation is clinical nephrology. Hercílio Martelli-Júnior and Enrico A. Colosimo were the recipients of CNPq scholarships in the areas of Dentistry and Mathematics, respectively.

Author information

Authors and Affiliations

  1. Pediatric Nephrology Unit, Federal University of Minas Gerais (UFMG), Belo Horizonte, MG, Brazil

    Eduardo Araujo Oliveira, Isabel Gomes Quirino, Maria Christina Lopes Oliveira, Mariana Guerra Duarte, Sergio Veloso Pinheiro & Ana Cristina Simões e Silva

  2. Nephrology Center, Pontífícia Universidade Católica (PUC, Curitiba), Curitiba, PR, Brazil

    Roberto Peicots-Filho

  3. Health Science Program, Universidade Estadual de Montes Claros, Montes Claros, MG, Brazil

    Daniella Reis Martelli & Hercílio Martelli-Júnior

  4. Department of Statistics, Federal University of Minas Gerais (UFMG), Belo Horizonte, MG, Brazil

    Enrico Antonio Colosimo

  5. Rua Engenheiro Amaro Lanari, 389/501, Belo Horizonte, MG, 30310-580, Brazil

    Eduardo Araujo Oliveira

Authors
  1. Eduardo Araujo Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Roberto Peicots-Filho
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniella Reis Martelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Isabel Gomes Quirino
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Maria Christina Lopes Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mariana Guerra Duarte
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sergio Veloso Pinheiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Enrico Antonio Colosimo
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Ana Cristina Simões e Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Hercílio Martelli-Júnior
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eduardo Araujo Oliveira.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Oliveira, E.A., Peicots-Filho, R., Martelli, D.R. et al. Is there a correlation between journal impact factor and researchers’ performance? A study comprising the fields of clinical nephrology and neurosciences. Scientometrics 97, 149–160 (2013). https://doi.org/10.1007/s11192-013-0992-x

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-013-0992-x

Keywords:

JEL Classification

Search

Navigation