Skip to main content
SpringerLink
Log in

Two simple new bibliometric indexes to better evaluate research in disciplines where publications typically receive less citations

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

The paper proposes two simple new indexes—k and w—to assess a scientist’s publications record based on citations. The two indexes are superior to the widely used h index (Hirsch, 2005), as they preserve all its valuable characteristics and try to overcome one of its shortcomings, i.e. that it uses only a fraction of the information contained in a scientist’s citations profile and, as a result, it is defined over the set of positive integers and does not show a sufficiently fine ‘granularity’ to allow a fully satisfactory ranking of scientists. This problem is particularly acute in many areas of Social Sciences and Humanities, where scientific productivity and citation practices typically yield fewer citations per paper and, as a consequence, are characterized by ‘structurally’ lower values of the h index. Both the indexes proposed are defined over R+, their integer part is equal to the scientist’s h index and they fall in the right-open interval [h, h+1). While the h index is influenced only by part of the citations received by a scientist’s most-cited publications, the k index takes into account all the citations received by her most-cited publications and the w index accounts for the citations received by the entire set of her publications. Variants of the k and w indexes are proposed which consider co-authorship. To show the extent to which the h index and the new indexes proposed may yield different results, they are calculated for 332 professors of economics in Italian universities and the results obtained used to rank Italian university departments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. In a non-ideal world, peer review too may have its limitations, including the possibility of highly subjective evaluations or conflicts of interest, and a bias against innovative ideas and approaches.

  2. A pertinent and thorough discussion of the use and misuse of citation statistics in quantitative indicators to assess scientific research performances is provided by Adler, Ewing and Taylor (2008).

  3. Nederhof (2006) provides a useful review of differences in publication and citation behaviors between many Social Sciences and Humanities and ‘hard’ sciences and discusses the implications of such differences for analyzing research performances based on bibliometric indexes.

  4. Any publication has a probability of receiving a given number of citations which increases with time since it appeared.

  5. Alonso et al. (2009) and Todeschini (2011) provide useful discussions of the pros and cons of using the h index and a review of variants to the original index which have been proposed to overcome some of its shortcomings.

  6. This is sometimes also referred to as the A index.

  7. Many more additional variants of the h index have been proposed, including those by Garcìa-Pérez (2009), Panaretos and Chrisovaladis (2009), Todeschini (2011) and Tol (2009).

  8. For a scientist whose h index equals k the h core is defined, after ranking his publications in descending order with respect to the number of citations received, as the subset of the first k of his publications which each received at least k citations, while the remaining ones each received at most k citations.

  9. Italian Ministry of Education, University and Research, Ministerial Decree no. 89/2009, 28 July, 2009.

  10. If w is preferred, scientists with a career extending longer into the past may have been given an advantage.

  11. They include ‘ricercatori’, ‘ricercatori non confermati’ and ‘ricercatori a tempo determinato’; ‘professori associati’ ‘confermati’ e ‘non confermati’; ‘professori ordinari’ e ‘straordinari’ active in the 64 Italian - public and officially accredited by Italian Ministry of Education, University and Research (MIUR) private universities—in which at least one professor of ‘Economic Policy’ is present. MIUR classifies professors in Economics in six groups: ‘Economics’, ‘Economic Policy’, ‘Public Finance’, ‘History of Economic Theory’, ‘Econometrics’ and ‘Applied Economics’. The set was extracted from the MIUR database (http://cercauniversita.cineca.it/php5/docenti/cerca.php) on 28th Feb 2012.

  12. Studies related to the evaluation of the research performance of professors of economics in Italian universities using bibliometric indexes include Checchi and Jappelli (2009), Corsi and Defrancesco (2012), Corsi, D’Ippoliti and Lucidi (2010, 2011), Lippi and Peracchi (2007), and Reichlin (2008). Checchi and Jappelli (2009) used the h index calculated using information from the Google Scholar data base to evaluate the 696 full professors of Economics. Corsi and Defrancesco (2012) analyzed the publication records of the 311 full, associate and assistant professors of ‘Agricultural Economics and Rural Appraisal’ calculating the h and g indexes based on information from the SCOPUS, ISI-WoK and Google Scholar (via Publish or Perish) data bases.

  13. No filtering based on the author’s affiliation has been performed, as it is impossible to control for the mobility of an author from one institution to another.

  14. The data base with the publications record from ISI-WoK, the citations profile and the bibliometric indexes for each of the 332 university professors of 'Economic Policy' of Italian universities, and a table with the values of the six indexes calculated for each of them are available at http://www.ecostat.unical.it/anania/AnaniaandCaruso.htm.

  15. An alternative way of measuring the performance of research institutions is the one based on the ‘successive h-index’ (h 2 ) proposed by Schubert (2007). An institution has an index h 2 equal to s if s of its n researchers each have a h index at least equal to s, and the remaining (n–s) each have a h index which does not exceed s.

  16. A table containing the average values of the h, k, w, h-norm, k-norm and w-norm indexes for the 109 departments, the rankings based on each of these indexes and the differences between these rankings is available at http://www.ecostat.unical.it/anania/AnaniaandCaruso.htm.

References

  • Adler, R., Ewing, J., & Taylor, P. (2008). Citation Statistics. A report from the International Mathematical Union (IMU) in cooperation with the International Council of Industrial and Applied Mathematics (ICIAM) and the Institute of Mathematical Statistics (IMS). (http://mfa-national.org/yahoo_site_admin/assets/docs/Documents/CitationStatistics-FINAL-1.pdf).

  • Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2009). H-index: A Review Focused in Its Variants, Computation and Standardization for Different Scientific Fields. Journal of Informetrics, 3(4), 273–289.

    Article  Google Scholar 

  • Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2010). Hg-index: A New Index to Characterize the Scientific Output of Researchers Based on the h- and g- Indices. Scientometrics, 82(2), 391–400.

    Article  Google Scholar 

  • Anderson, T. R., Hankin, R. K., & Killworth, P. D. (2008). Beyond the Durfee square: Enhancing the h-index to score total publication output. Scientometrics, 76(3), 577–588.

    Article  Google Scholar 

  • Batista, P. D., Campiteli, M. G., Kinouchi, O., & Martinez, A. S. (2006). Is it possible to compare researchers with different scientific interests? Scientometrics, 68(1), 179–189.

    Article  Google Scholar 

  • Bornmann, L., Mutz, R., & Daniel, H.-D. (2008). Are There Better Indices for Evaluation Purposes than the h Index? A Comparison of Nine Different Variants of the h index Using Data from Biomedicine. Journal of the American Society for Information Science and Technology, 59(5), 830–837.

    Article  Google Scholar 

  • Checchi, D., & Jappelli, T. (2008). Ricerca per indice h. La Voce, 16 dicembre (www.lavoce.info).

  • Corsi, M., D’Ippoliti, C., & Lucidi, F. (2010). Pluralism at Risk? Heterodox Economic Approaches and the Evaluation of Economic Research in Italy. American Journal of Economics and Sociology, 69(5), 1495–1529.

    Article  Google Scholar 

  • Corsi, M., D’Ippoliti, C., & Lucidi, F. (2011). On the Evaluation of Economic Research: The Case of Italy. Economia Politica, 28(3), 369–402.

    Google Scholar 

  • Corsi, A., & Defrancesco, E. (2012). La valutazione basata su peer-review e indicatori bibliometrici: esperienze in atto e riflessioni sul settore AGR/01. Agriregionieuropa 8(28) 31–36 (http://agriregionieuropa.univpm.it/).

    Google Scholar 

  • Egghe, L. (2006). Theory and practice of the g-index. Scientometrics, 69(1), 131–152.

    Article  MathSciNet  Google Scholar 

  • Garcìa-Pérez, M. A. (2009). A multidimensional extension to Hirsch’s h-index. Scientometrics, 81(3), 779–785.

    Article  Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.

    Article  Google Scholar 

  • Jin, B. H. (2006). h index: An evaluation indicator proposed by scientist. Science Focus (in Chinese), 1(1), 8–9.

    Google Scholar 

  • Jin, B. H., Liang, L. M., Rousseau, R., & Egghe, L. (2007). The R- and AR-indices: Complementing the h-index. Chinese Science Bulletin, 52(6), 855–863.

    Article  Google Scholar 

  • Kosmulski, M. (2006). A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newsletter, 2(3), 4–6.

    Google Scholar 

  • Lippi, M., & Peracchi, F. (2007). Il primo esercizio italiano di valutazione della ricerca: una prima valutazione. Rivista Italiana degli Economisti, 12(2), 267–276.

    Google Scholar 

  • Nederhof, A. M. (2006). Bibliometric monitoring of research performance in the Social Sciences and Humanities: A review. Scientometrics, 66(1), 81–100.

    Article  MathSciNet  Google Scholar 

  • Panaretos, J., & Chrisovaladis, M. (2009). Assessing scientific research performance and impact with single indices. Scientometrics, 81(3), 635–670.

    Article  Google Scholar 

  • Reichlin, P. (2008). La valutazione della ricerca in economia. Statistica & Società, 6 (special issue), 33–38.

  • Ruane, F. P., & Tol, R. S. J. (2008). Rational (successive) h-indices: An application to economics in the Republic of Ireland. Scientometrics, 75(2), 395–405.

    Article  Google Scholar 

  • Schreiber, M. (2008). To share the fame in a fair way, hm modifies h for multi-authored manuscripts. New Journal of Physics 10(4) (open access, Doi: 10.1088/1367-2630/10/4/040201).

  • Schubert, A. (2007). Successive h-indices. Scientometrics, 70(1), 201–205.

    Article  Google Scholar 

  • Todeschini, R. (2011). The j-index: a new bibliometric index and multivariate comparisons between other common indices. Scientometrics, 87(3), 621–639.

    Article  Google Scholar 

  • Tol, R. S. J. (2009). The h-index and its alternatives: An application to the 100 most prolific economists. Scientometrics, 80(2), 317–324.

    Article  Google Scholar 

  • Zhang, C.-T. (2009). The e-index, Complementing the h-index for Excess Citations. Plos One 4(5), 1–4 May. (www.plosone.org).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Economics, Statistics and Finance, University of Calabria, Rende, Italy

    Giovanni Anania & Annarosa Caruso

Authors
  1. Giovanni Anania
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Annarosa Caruso
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Giovanni Anania.

Additional information

We wish to thank Giuseppe Rose and conference participants at the annual meetings of the Associazione Italiana di Economia Agraria ed Applicata (Trento, 4–5 June 2012) and the Società Italiana degli Economisti (Matera, 18–20 Oct 2012) for helpful comments on an earlier draft of the paper.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Anania, G., Caruso, A. Two simple new bibliometric indexes to better evaluate research in disciplines where publications typically receive less citations. Scientometrics 96, 617–631 (2013). https://doi.org/10.1007/s11192-013-0951-6

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-013-0951-6

Keywords

JEL Classification

Search

Navigation