Skip to main content
SpringerLink
Log in

Exploring the limitations of the h-index and h-type indexes in measuring the research performance of authors

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

With the introduction of an increasing number of evaluation indexes, researchers have begun to pay attention to the limitations of such indexes in research evaluation, understanding which to avoid misusing and making evaluation more scientific and reasonable. Analysing the principles of the h-index, g-index, AR-index, p-index, integrated impact indicator (I3), and academic trace, this paper explores their limitations in measuring the research performance of authors from the perspectives of consistency, the degree of discrimination, and the statistical relationship between the values of indicators and the number of publications and citations. There are some interesting findings. These six indicators are highly consistent, and they are all more susceptible to the number of publications than to the frequency of citations. Among them, the h-index has the lowest degree of discrimination, followed by the g-index, I3, AR-index, p-index, and academic trace. The g-index ignores papers and citations other than the g-core. Moreover, compared to the h-index, the accumulation of citations makes it easier for the g-index to be equal to the number of papers published by an author, and once its value equals the number of papers, subsequent citations received by these papers will no longer contribute to the growth of the g-index unless the author publishes a new paper. Additionally, the AR-index ignores the h-tail papers and citations, which underestimates the impact of many researchers. Moreover, the p-index is insensitive to highly cited papers. Furthermore, the I3 is very vulnerable to the influence of the extremums in a data set. Finally, we propose considerations and suggestions for the research performance evaluation of authors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Abramo, G., D’Angelo, C. A., & Viel, F. (2013a). Assessing the accuracy of the h- and g-indexes for measuring researchers’ productivity. Journal of the American Society for Information Science and Technology,64(6), 1224–1234.

    Google Scholar 

  • Abramo, G., D’Angelo, C. A., & Viel, F. (2013b). The suitability of h and g indexes for measuring the research performance of institutions. Scientometrics,97(3), 555–570.

    Google Scholar 

  • Agarwal, A., Durairajanayagam, D., Tatagari, S., et al. (2016). Bibliometrics -tracking research impact by selecting the appropriate metrics. Asian Journal of Andrology,18(2), 296–309.

    Google Scholar 

  • Ajiferuke, I., Lu, K., & Wolfram, D. (2010). A comparison of citer and citation-based measure outcomes for multiple disciplines. Scientometrics,61(10), 2086–2096.

    Google Scholar 

  • Ajiferuke, I., & Wolfram, D. (2010). Citer analysis as a measure of research impact: Library and information science as a case study. Scientometrics,83(3), 623–638.

    Google Scholar 

  • Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., et al. (2010). Hg-index: A new index to characterize the scientific output of researchers based on the h-and g-indices. Scientometrics,82(2), 391–400.

    Google Scholar 

  • Ball, P. (2005). Index aims for fair ranking of scientists. Nature,436(7053), 900.

    Google Scholar 

  • Barnes, C. (2017). The h-index debate: An introduction for librarians. Journal of Academic Librarianship,43(6), 487–494.

    Google Scholar 

  • Batista, P. D., Campiteli, M. G., & Kinouchi, O. (2006). Is it possible to compare researchers with different scientific interests? Scientometrics,68(1), 179–189.

    Google Scholar 

  • Bornmann, L. (2013). How to analyze percentile citation impact data meaningfully in bibliometrics: The statistical analysis of distributions, percentile rank classes, and top-cited papers. Journal of the American Society for Information Science and Technology,64(3), 587–595.

    Google Scholar 

  • Bornmann, L. (2014). H-Index research in scientometrics: A summary. Journal of Informetrics,3(8), 749–750.

    Google Scholar 

  • Bornmann, L., Mutz, R., Hug, S., et al. (2011). A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants. Journal of Informetrics,5(3), 346–359.

    Google Scholar 

  • Bornmann, L., Rüdiger, M., & Daniel, H. D. (2008). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology,59(5), 830–837.

    Google Scholar 

  • Burrell, Q. L. (2007). On the h-index, the size of the Hirsch core and Jin’s A-index. Journal of Informetrics,1(2), 170–177.

    MathSciNet  Google Scholar 

  • Costas, R., & Bordons, M. (2008). Is g-index better than h-index? An exploratory study at the individual level. Scientometrics,77(2), 267–288.

    Google Scholar 

  • Egghe, L. (2006). Theory and practice of the g-index. Scientometrics,69(1), 131–152.

    MathSciNet  Google Scholar 

  • Egghe, L. (2010). The Hirsch index and related impact measures. Annual Review of Information Science and Technology,44(1), 65–114.

    Google Scholar 

  • Garfield, E. (1955). Citation indexes for science. Science,122(3159), 108–111.

    Google Scholar 

  • Gingras, Y., & Larivière, V. (2011). There are neither “king” nor “crown” in scientometrics: Comments on a supposed “alternative” method of normalization. Journal of Informetrics,5(1), 226–227.

    Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceeding of the National Academy of Sciences of USA,102(46), 16569–16572.

    MATH  Google Scholar 

  • Hirsch, J. E. (2010). An index to quantify an individual’s scientific research output that takes into account the effect of multiple coauthorship. Scientometrics,85(3), 741–754.

    Google Scholar 

  • Hirsch, J. E. (2019). hα: An index to quantify an individual’s scientific leadership. Scientometrics,118(2), 673–686.

    Google Scholar 

  • Iglesias, J. E., & Pecharromán, C. (2007). Scaling the h-index for different scientific ISI fields. Scientometrics,73(3), 303–320.

    Google Scholar 

  • Jin, B. H. (2006). H-index: An evaluation indicator proposed by scientist. Science Focus (in Chinese),1(1), 8–9.

    Google Scholar 

  • Jin, B. H., Liang, L. M., Rousseau, R., et al. (2007). The R-and AR indices: Complementing the h-index. Chinese Science Bulletin,52(6), 855–863.

    Google Scholar 

  • Kozak, M., & Bornmann, L. (2012). A new family of cumulative indexes for measuring scientific performance. PLoS ONE,7(10), e47679.

    Google Scholar 

  • Leydesdorff, L., & Bornmann, L. (2011). Integrated impact indicators compared with impact factors: An alternative research design with policy implications. Journal of the American Society for Information Science and Technology,62(11), 2133–2146.

    Google Scholar 

  • Liang, L. (2006). h-index sequence and h-index matrix: Constructions and applications. Scientometrics,69(1), 153–159.

    Google Scholar 

  • Prathap, G. (2010). Is there a place for a mock h-index? Scientometrics,84(1), 153–165.

    Google Scholar 

  • Rousseau, R. (2006). Simple models and the corresponding h- and g-index. Retrieved March 11, 2019, from http://eprints.rclis.org/7501/1/Rousseau_Dalian.pdf.

  • Rousseau, R., & Ye, F. Y. (2012). A formal relation between the h-index of a set of articles and their I3 score. Journal of Informetrics,6(1), 34–35.

    Google Scholar 

  • Schreiber, M. (2008a). An empirical investigation of the g-index for 26 physicists in comparison with the h-index, the A-index, and the R-index. Journal of the American Society for Information Science and Technology,59(9), 1513–1522.

    Google Scholar 

  • Schreiber, M. (2008b). To share the fame in a fair way, hm modifies h for multi-authored manuscripts. New Journal of Physics,10(4), 1–9.

    MathSciNet  Google Scholar 

  • Schreiber, M. (2009). Fractionalized counting of publications for the g-Index. Journal of the American Society for Information Science and Technology,60(10), 2145–2150.

    Google Scholar 

  • Schreiber, M. (2010a). Twenty Hirsch index variants and other indicators giving more or less preference to highly cited papers. Annalen der Physik,522(8), 536–554.

    Google Scholar 

  • Schreiber, M. (2010b). Revisiting the g-index: The average number of citations in the g-core. Journal of the American Society for Information Science and Technology,61(1), 169–174.

    Google Scholar 

  • Sidiropoulos, A., Katsaros, D., & Manolopoulos, Y. (2007). Generalized Hirsch h-index for disclosing latent facts in citation networks. Scientometrics,72(2), 253–280.

    Google Scholar 

  • Tol, R. S. J. (2008). A rational, successive g-index applied to economics departments in Ireland. Journal of Informetrics,2(2), 149–155.

    Google Scholar 

  • Wagner, C. S., & Leydesdorff, L. (2012). An Integrated Impact Indicator: A new definition of ‘Impact’ with policy relevance. Research Evaluation,21(3), 183–188.

    Google Scholar 

  • Woeginger, G. J. (2009). Generalizations of Egghe’s g-index. Journal of the American Society for Information Science and Technology,60(6), 1267–1273.

    Google Scholar 

  • Ye, F. Y. (2014). Overview of research status and development of international academic evaluation indicators. Journal of Intelligence,33(2), 215–223.

    Google Scholar 

  • Ye, F. Y., Bornmann, L., & Leydesdorff, L. (2017). h-based I3-type multivariate vectors: Multidimensional indicators of publication and citation scores. COLLNET Journal of Scientometrics and Information Management,11(1), 153–171.

    Google Scholar 

  • Ye, F. Y., & Leydesdorff, L. (2014). The “Academic Trace” of the performance matrix: A mathematical synthesis of the h-index and the integrated impact indicator (I3). Journal of the Association for Information Science and Technology,65(4), 742–750.

    Google Scholar 

  • Zhang, C. T. (2009a). The e-index, complementing the h-index for excess citations. PLoS ONE,4(5), e5429.

    Google Scholar 

  • Zhang, C. T. (2009b). A proposal for calculating weighted citations based on author rank. EMBO Reports,10(5), 416–417.

    Google Scholar 

  • Zhang, C. T. (2013). The h’-index, effectively improving the h-index based on the citation distribution. PLoS ONE,8(4), e59912.

    Google Scholar 

Download references

Acknowledgements

We wish to thank the anonymous referees for important insightful comments and suggestions. This research was funded by Project of the National Social Science Foundation of China (Grant No. 17BTQ071).

Author information

Authors and Affiliations

  1. Department of Library, Information and Archives, Shanghai University, Shanghai, China

    Jingda Ding, Chao Liu & Goodluck Asobenie Kandonga

Authors
  1. Jingda Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Goodluck Asobenie Kandonga
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JD was involved in conceptualization, formal analysis, methodology, supervision, review, the preparation of the initial draft, and the editing of the final draft. CL was involved in data collection, formal analysis, methodology, software, and the preparation of the initial draft. GK was involved in formal analysis, review and the editing of the final draft.

Corresponding author

Correspondence to Jingda Ding.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ding, J., Liu, C. & Kandonga, G.A. Exploring the limitations of the h-index and h-type indexes in measuring the research performance of authors. Scientometrics 122, 1303–1322 (2020). https://doi.org/10.1007/s11192-020-03364-1

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-020-03364-1

Keywords

Search

Navigation