Abstract
Citation count fails to comprehensively and accurately portray the publication’s impact, and the scientific impact is far more than what we see on the surface. This paper proposes a new impact indicator, the Contribution-Weighted Citation index, in which each citation is weighted from the perspective of the citing item. The method of weighting is based on ranking of the ‘old’ impact values of articles published in the same year and same field, and modifies the value of one citation from each publication positively or negatively. The idea of iteration is introduced into the calculation process for more accurate results. We empirically analysed 3,847,243 papers from the WoS database for the 1978–2017 period, including the fields of Mathematics, Physics, and Space Science. The experimental results show that the three variants of the new index proposed in this article show different characteristics in identifying article impact. Our new indicator in sine mode make articles more discrete, which helps distinguish between papers with similar citation count, while the arcsine mode is more helpful to identify high-citation-quality papers. In addition, we found the average citation quality of lowly cited papers fluctuates greatly, and some such papers have surprisingly high citation quality; for highly cited case, the average citation quality stabilizes in a certain range. This article has a positive role in identifying important publications and under-cited publications.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abramo, G., Cicero, T., & D’Angelo, C. A. (2012). Revisiting the scaling of citations for research assessment. Journal of Informetrics, 6(4), 470–479. https://doi.org/10.1016/j.joi.2012.03.005
Bai, X., Feng, X., Lee, I., Zhang, J., & Ning, Z. (2016). Identifying anomalous citations for objective evaluation of scholarly article impact. PLoS ONE, 11(9), 1–15. https://doi.org/10.1371/journal.pone.0162364
Berger, S., Stocker, H., & Zeileis, A. (2016). Innovation and institutional ownership revisited: An empirical investigation with count data models. Empirical Economics, 52(4), 1675–1688. https://doi.org/10.1007/s00181-016-1118-0
Bollen, J., Rodriquez, M. A., & Van De Sompel, H. (2006). Journal status. Scientometrics, 69(3), 669–687. https://doi.org/10.1007/s11192-006-0176-z
Buela-Casal, G. (2004). Assessing the quality of articles and scientific journals: Proposal for weighted impact factor and a quality index. Psychology in Spain, 8(1), 60–76.
Cai, L., Tian, J., Liu, J., et al. (2019). Scholarly impact assessment: A survey of citation weighting solutions. Scientometrics, 118(2), 453–478. https://doi.org/10.1007/s11192-018-2973-6
Chen, P., Xie, H., Maslov, S., & Redner, S. (2007). Finding scientific gems with Google’s PageRank algorithm. Journal of Informetrics, 1(1), 8–15. https://doi.org/10.1016/j.joi.2006.06.001
Davis, P. M. (2008). Eigenfactor: Does the principle of repeated improvement result in better estimates than raw citation counts? Journal of the American Society for Information Science and Technology, 59(13), 2186–2188. https://doi.org/10.1002/asi.20943
Dervos, D. A., & Kalkanis, T. (2005). Cc-IFF: A cascading citations impact factor framework for the automatic ranking of research publications. In 2005 IEEE Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, Sofia, 2005, (pp 668–673). https://doi.org/10.1109/IDAACS.2005.283070
Ding, Y., & Cronin, B. (2011). Popular and/or prestigious? Measures of scholarly esteem. Information Processing & Management, 47(1), 80–96. https://doi.org/10.1016/j.ipm.2010.01.002
Egghe, L. (2010). On the relation between Schubert’s h-index of a single paper and its total number of received citations. Scientometrics, 84(1), 115–117. https://doi.org/10.1007/s11192-009-0062-6
Egghe, L., Bornmann, L., & Guns, R. (2011). A proposal for a First-Citation-Speed-Index. Journal of Informetrics, 5(1), 181–186. https://doi.org/10.1016/j.joi.2010.10.006
Fiala, D. (2012). Bibliometric analysis of CiteSeer data for countries. Information Processing & Management, 48(2), 242–253. https://doi.org/10.1016/j.ipm.2011.10.001
Fiala, D., Rousselot, F., & Ježek, K. (2008). PageRank for bibliographic networks. Scientometrics, 76(1), 135–158. https://doi.org/10.1007/s11192-007-1908-4
Fragkiadaki, E., & Evangelidis, G. (2014). Review of the indirect citations paradigm: Theory and practice of the assessment of papers, authors and journals. Scientometrics, 99(2), 261–288. https://doi.org/10.1007/s11192-013-1175-5
Franceschet, M. (2010). The difference between popularity and prestige in the sciences and in the social sciences: A bibliometric analysis. Journal of Informetrics, 4(1), 55–63. https://doi.org/10.1016/j.joi.2009.08.001
Garfield, E. (1955). Citation indexes for science: A new dimension in documentation through association of ideas. Science, 122(3159), 108–111. https://doi.org/10.1126/science.122.3159.108
Giuffrida, C., Abramo, G., & D’Angelo, C. A. (2019). Are all citations worth the same? Valuing citations by the value of the citing items. Journal of Informetrics, 13(2), 500–514. https://doi.org/10.1016/j.joi.2019.02.008
Hammarfelt, B. (2011). Citation analysis on the micro level : The example of Walter Benjamin’s illuminations. Journal of the American Society for Information Science and Technology, 62(5), 819–830. https://doi.org/10.1002/asi.21504
Hu, X., & Rousseau, R. (2016). Scientifc infuence is not always visible: The phenomenon of under-cited infuential publications. Journal of Informetrics, 10(4), 1079–1091. https://doi.org/10.1016/j.joi.2016.10.002
Hu, X., Rousseau, R., & Chen, J. (2011). On the definition of forward and backward citation generations. Journal of Informetrics, 5(1), 27–36. https://doi.org/10.1016/j.joi.2010.07.004
Jiang, X., & Zhuge, H. (2019). Forward search path count as an alternative indirect citation impact indicator. Journal of Informetrics, 13(4), 100977. https://doi.org/10.1016/j.joi.2019.100977
Ketzler, R., & Zimmermann, K. F. (2013). A citation-analysis of economic research institutes. Scientometrics, 95(3), 1095–1112. https://doi.org/10.1007/s11192-012-0850-2
Klavans, R., & Boyack, K. W. (2017). Which type of citation analysis generates the most accurate taxonomy of scientific and technical knowledge ? Journal of the American Society for Information Science and Technology, 68(4), 984–998. https://doi.org/10.1002/asi.23734
Li, J., Wu, D., Li, J., & Li, M. (2017). A comparison of 17 article-level bibliometric indicators of institutional research productivity: Evidence from the information management literature of China. Information Processing & Management, 53(5), 1156–1170. https://doi.org/10.1016/j.ipm.2017.05.002
Lundberg, J. (2007). Lifting the crown-citation z-score. Journal of Informetrics, 1(2), 145–154. https://doi.org/10.1016/j.joi.2006.09.007
Margolis, J. (1967). Citation indexing and evaluation of scientific papers. Science, 155(3767), 1213–1219. https://doi.org/10.1126/science.155.3767.1213
Maslov, S., & Redner, S. (2008). Promise and pitfalls of extending Google’s PageRank algorithm to citation networks. Journal of Neuroscience, 28(44), 11103–11105. https://doi.org/10.1523/JNEUROSCI.0002-08.2008
Nejati, A., & Hosseini Jenab, S. M. (2010). A two-dimensional approach to evaluate the scientific production of countries (case study: The basic sciences). Scientometrics, 84(2), 357–364. https://doi.org/10.1007/s11192-009-0103-1
Pinski, G., & Narin, F. (1976). Citation influence for journal aggregates of scientific publications: Theory, with application to the literature of physics. Information Processing & Management, 12(5), 297–312. https://doi.org/10.1016/0306-4573(76)90048-0
Rousseau, R. (1987). The Gozinto theorem: Using citations to determine influences on a scientific publication. Scientometrics, 11(3–4), 217–229. https://doi.org/10.1007/BF02016593
Ruiz-Castillo, J. (2014). The comparison of classification-system-based normalization procedures with source normalization alternatives in Waltman and Van Eck (2013). Journal of Informetrics, 8(1), 25–28. https://doi.org/10.1016/j.joi.2013.10.002
Schubert, A. (2009). Using the h-index for assessing single publications. Scientometrics, 78(3), 559–565. https://doi.org/10.1007/s11192-008-2208-3
Su, C., Pan, Y. T., Zhen, Y. N., Ma, Z., Yuan, J. P., Guo, H., et al. (2011). PrestigeRank: A new evaluation method for papers and journals. Journal of Informetrics, 5(1), 1–13. https://doi.org/10.1016/j.joi.2010.03.011
Vaccario, G., Medo, M., Wider, N., & Mariani, M. S. (2017). Quantifying and suppressing ranking bias in a large citation network. Journal of Informetrics, 11(3), 766–782. https://doi.org/10.1016/j.joi.2017.05.014
Walker, D., Xie, H., Yan, K. K., & Maslov, S. (2007). Ranking scientific publications using a model of network traffic. Journal of Statistical Mechanics, 06, P06010. https://doi.org/10.1088/1742-5468/2007/06/P06010
Waltman, L. (2016). A review of the literature on citation impact indicators. Journal of Informetrics, 10(2), 365–391. https://doi.org/10.1016/j.joi.2016.02.007
Waltman, L., & Van Eck, N. J. (2013). Source normalized indicators of citation impact: An overview of different approaches and an empirical comparison. Scientometrics, 96(3), 699–716. https://doi.org/10.1007/s11192-012-0913-4
Yan, E., & Ding, Y. (2010). Weighted citation: An indicator of an article’s prestige. Journal of the American Society for Information Science and Technology, 61(8), 1635–1643. https://doi.org/10.1002/asi.21349
Yan, E., & Sugimoto, C. R. (2011). Institutional interactions: Exploring social, cognitive, and geographic relationships between institutions as demonstrated through citation networks. Journal of the American Society for Information Science and Technology, 62(8), 1498–1514. https://doi.org/10.1002/asi.21556
Yan, E., Ding, Y., & Sugimoto, C. R. (2011). P-Rank: An indicator measuring prestige in heterogeneous scholarly networks. Journal of the American Society for Information Science and Technology, 62(3), 467–477. https://doi.org/10.1002/asi.21461
Zhu, X., Turney, P., Lemire, D., & Vellino, A. (2015). Measuring academic influence : Not all citations are equal. Journal of the American Society for Information Science and Technology, 66(2), 408–427. https://doi.org/10.1002/asi.23179
Zitt, M., & Small, H. (2008). Modifying the journal impact factor by fractional citation weighting: The audience factor. Journal of the American Society for Information Science and Technology, 59(11), 1856–1860. https://doi.org/10.1002/asi.20880
Acknowledgements
This research was supported by the National Natural Science Foundation of China (Grant No. 71874173) and the Academic Division of Mathematics and Physics of the Chinese Academy of Sciences (Grant No. 2018-M04-B-004). We would like to thank the editor and anonymous reviewers for their constructive comments and suggestions, which helped us to improve the paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Liu, Y., Wu, Q., Wu, S. et al. Weighted citation based on ranking-related contribution: a new index for evaluating article impact. Scientometrics 126, 8653–8672 (2021). https://doi.org/10.1007/s11192-021-04115-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-021-04115-6