Skip to main content
Log in

Comparison of Brazilian researchers in clinical medicine: are criteria for ranking well-adjusted?

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Quantifying the relative performance of individual scholars has become an integral part of decision-making in research policy. The objective of the present study was to evaluate if the scholarship rank of Brazilian Council for Scientific and Technological Development (CNPq) researchers in Medicine is consistent with their scientific productivity. The Lattes curricula of 411 researchers (2006–2008) were included in the study. Scholarship category was the variable of interest. Other variables analyzed were: time since receiving the doctorate, teaching activity (undergraduate, master’s and doctoral students), number of articles published, and number of papers indexed by the Institute for Scientific Information (ISI) and Scopus databases. Additional performance indicators included were: citations, h-index, and m-index. There was a significant difference among scholarship categories regarding number of papers per year, considering the entire scientific career (P < 0.001) or the last 5 years (P < 0.001). There was no significant difference among scholarship categories regarding the number of citations per article in the ISI (Thomson Reuters) database (P = 0.23). There was a significant difference in h-index among scholarship categories in both databases, i.e. (P < 0.001) and Scopus (P < 0.001). Regarding the m-index, there was a significant difference among categories only in the ISI database (P = 0.012). According to our findings, a better instrument for qualitative and quantitative indicators is needed to identify researchers with outstanding scientific output.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Abramo, G., D’Angelo, C. A., & Costa, F. (2011a). National research assessment exercises: The effects of changing the rules of the game during the game. Scientometrics, 88, 229–238.

    Article  Google Scholar 

  • Abramo, G., D’Angelo, C. A., & Costa, F. (2011b). Research productivity: Are higher academic ranks more productive than lower ones? Scientometrics. doi:10.1007/s11192-011-0426-6.

  • Allen, L., Jones, C., Dolby, K., Lynn, D., & Walport, M. (2009). Looking for landmarks: The role of expert review and bibliometric analysis in evaluating scientific publication outputs. PLoS One, 4(6), e5910. doi:10.1371/journal.pone.0005910.

    Article  Google Scholar 

  • Arruda, D., Bezerra, F., Neris, V. A., Toro, P. R., & Wainer, J. (2009). Brazilian computer science research: Gender and regional distributions. Scientometrics, 79(3), 651–665.

    Article  Google Scholar 

  • Barata, R. B., & Goldbaum, M. (2003). A profile of researchers in public health with productivity grants from the Brazilian National Research Council (CNPq). Cad Saude Publica, 19(6), 1863–1876.

    Article  Google Scholar 

  • Boell, S. K., & Wilson, C. S. (2010). Journal impact factors for evaluating scientific performance: Use of h-like indicators. Scientometrics, 82, 613–626.

    Article  Google Scholar 

  • Bornmann, L., & Daniel, H.-D. (2007). What do we know about the h index? Journal of the American Society for Information Science, 58, 1381–1385.

    Article  Google Scholar 

  • Bornmann, L., Mutz, R., & Daniel, H.-D. (2008a). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science & Technology, 59(5), 830–837.

    Article  Google Scholar 

  • Bornmann, L., Mutz, R., Neuhaus, C., & Daniel, H.-D. (2008b). Citation counts for research evaluation: Standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics in Science and Environmental Politics, 8, 93–102.

    Article  Google Scholar 

  • Bornmann, L., Wallon, G., & Ledin, A. (2008c). Is the h index related to (standard) bibliometric measures and to the assessments by peers? An investigation of the h index by using molecular life sciences data. Research Evaluation, 17(2), 149–156.

    Article  Google Scholar 

  • Browman, H. I., & Stergiou, K. I. (2008). Factors and indices are one thing, deciding who is scholarly, why they are scholarly, and the relative value of their scholarship is something else entirely. Ethics in Science and Environmental Politics, 8, 1–3.

    Article  Google Scholar 

  • Burrell, Q. L. (2007a). Hirsch index or Hirsch rate? Some thoughts arising from Liang’s data. Scientometrics, 73(1), 19–28.

    Article  Google Scholar 

  • Burrell, Q. L. (2007b). Hirsch’s h-index: A stochastic model. Journal of Informetrics, 1(1), 16–25.

    Article  Google Scholar 

  • Cavalcante, R. A., Barbosa, D. R., Bonan, P. R. F., Pires, M. B. O., & Martelli-Junior, H. (2008). Perfil dos pesquisadores da área de odontologia no Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq). Revista Brasileira de Epidemiologia, 11(1), 106–113.

    Article  Google Scholar 

  • CNPq, Ministério de Ciência e Tecnologia. (2010). Critérios de Julgamento—CA-MD. http://www.cnpq.br/cas/ca-md.htm#criterios. Accessed 4th Aug 2010.

  • Franceschini, F., & Maisano, D. (2011). Proposals for evaluating the regularity of a scientist’s research output. Scientometrics, 88, 279–295.

    Article  Google Scholar 

  • Geuna, A., & Martin, B. R. (2003). University research evaluation and funding: An international comparison. Minerva, 41, 277–304.

    Article  Google Scholar 

  • Glanzel, W. (2006). On the h-index: A mathematical approach to a new measure of publication activity and citation impact. Scientometrics, 67, 315–321.

    Article  Google Scholar 

  • Glänzel, W., & Schubert, A. (2003). A new classification scheme of science fields and subfields designed for scientometric evaluation purposes. Scientometrics, 56, 357–367.

    Article  Google Scholar 

  • Haeffner-Cavaillon, N., & Graillot-Gak, C. (2009). The use of bibliometric indicators to help peer-review assessment. Archivum Immunologiae et Therapiae Experimentalis (Warsz), 57(1), 33–38.

    Article  Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.

    Article  Google Scholar 

  • Jeang, K. T. (2008). H-index, mentoring-index, highly-cited and highly-accessed: How to evaluate scientists? Retrovirology, 5, 106.

    Article  Google Scholar 

  • Lane, J. (2010). Let’s make science metrics more scientific. Nature, 464(7288), 488–489.

    Article  Google Scholar 

  • Lehmann, S., Jackson, A. D., & Lautrup, B. E. (2006). Measures for measures. Nature, 444(7122), 1003–1004.

    Article  Google Scholar 

  • Leite, P., Mugnaini, R., & Leta, J. (2011). A new indicator for international visibility: Exploring Brazilian scientific community. Scientometrics, 88, 311–319.

    Article  Google Scholar 

  • Martelli-Junior, H., Martelli, D. R., Quirino, I. G., Oliveira, M. C., Lima, L. S., & Oliveira, E. A. (2010). CNPq researchers in medicine: A comparative study of research areas. Revista Da Associacao Medica Brasileira, 56(4), 478–483.

    Article  Google Scholar 

  • Mendes, P. H. C., Martelli, D. R., Souza, W. P., Filho, S. Q., & Martelli Junior, H. (2010). Perfil dos pesquisadores bolsistas de produtividade científica na medicina no CNPq, Brasil. Revista Brasileira de Educação Médica (in press).

  • Moed, H. F. (2009). New developments in the use of citation analysis in research evaluation. Arch Immunol Ther Exp (Warsz), 57(1), 13–18. doi:10.1007/s00005-009-0001-5.

    Article  Google Scholar 

  • Montgomery, D. C. (2008). Design and analysis of experiments (Vol. 1, 7th ed.). Boston: Wiley.

  • Oliveira, E. A., Pecoits-Filho, R., Quirino, I. G., Oliveira, M. C. L., Martelli, D. R., Lima, L. S., et al. (2011a). Perfil e produção científica dos pesquisadores do CNPq nas áreas de Nefrologia e Urologia. Jornal Brasileiro de Nefrologia, 33, 31–37.

    Article  Google Scholar 

  • Oliveira, E. A., Ribeiro, A. L. P., Quirino, I. G., Oliveira, M. C., Martelli, D. R., Lima, L. S., et al. (2011b). Perfil e produção científica dos pesquisadores do Conselho Nacional de Desenvolvimento Científico e Tecnológico na área de Cardiologia. Arquivos Brasileiro de Cardiologia. doi:10.1590/S0066-782X2011005000086

  • Panaretos, J., & Malesios, C. (2009). Assessing scientific research performance and impact with single indices. Scientometrics, 81(3), 635–670.

    Article  Google Scholar 

  • Petherick, A. (2010). High hopes for Brazilian science. Nature, 465(7299), 674–675.

    Article  Google Scholar 

  • Podlubny, I. (2005). Comparison of scientific impact expressed by the number of citations in different fields of science. Scientometrics, 64, 95–99.

    Article  Google Scholar 

  • Regalado, A. (2010). Science in Brazil. Brazilian science: Riding a gusher. Science, 330(6009), 1306–1312. doi:10.1126/science.330.6009.1306.

    Google Scholar 

  • Santos, N. C. F., Candido, L. F. O., & Kuppens, C. L. (2010). Produtividade em pesquisa do CNPq: Análise do perfil dos pesquisadores da química. Quimica Nova, 33(2), 489–495.

    Article  Google Scholar 

  • Santos, S. M. C., Lima, L. S., Martelli, D. R. B., & Martelli Junior, H. (2009). Perfil dos pesquisadores da Saúde Coletiva no Conselho Nacional de Desenvolvimento Científico e Tecnológico. Physis Revista de Saúde Coletiva, 19(3), 761–775.

    Article  Google Scholar 

  • Shibayama, S. (2011). Distribution of academic research funds: A case of Japanese national research grant. Scientometrics, 88, 43–60.

    Article  Google Scholar 

  • van Raan, A. F. J. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics, 67, 491–502.

    Google Scholar 

Download references

Acknowledgements

This study was partially supported by CNPq (National Council for Scientific and Technological Development) and FAPEMIG (Research Support Foundation of Minas Gerais). Eduardo A. Oliveira and Ana Cristina Simões e Silva were the recipients of CNPq scholarships Category 2 in the area of Medicine. Hercílio Martelli-Júnior and Enrico A. Colosimo were the recipients of CNPq scholarships Category 2 in the areas of Dentistry and Mathematics, respectively.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Eduardo A. Oliveira.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Oliveira, E.A., Colosimo, E.A., Martelli, D.R. et al. Comparison of Brazilian researchers in clinical medicine: are criteria for ranking well-adjusted?. Scientometrics 90, 429–443 (2012). https://doi.org/10.1007/s11192-011-0492-9

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-011-0492-9

Keywords

JEL Classification

Navigation