Abstract
Publishing at high-rank journals is a common objective to most researchers, and there’s a crucial need for a journal ranking system with universal recognition. This paper presents a quantitative approach to rank scientific journals. The approach, HR-PageRank, combines weighted PageRank according to author’s H-index, and relevance between citing and cited papers. The output of the proposed approach is compared against journal impact factor, H5-index, PageRank algorithm and China Computer Federation ranking list. The experiments of quantifying scholarly impact objectively are conducted in two real scholarly data sets: (1) Microsoft Academic Graph and (2) Digital Bibliography and Library Project. Our experimental results indicate that HR-PageRank algorithm outperforms the well-known PageRank algorithm in finding the influential journals according to Spearman’s rank correlation coefficient, discounted cumulated gain and the correlation C.
Similar content being viewed by others
References
Agarwal, A., Durairajanayagam, D., Tatagari, S., Esteves, S. C., Harlev, A., Henkel, R., et al. (2016). Bibliometrics: Tracking research impact by selecting the appropriate metrics. Asian Journal of Andrology, 18(2), 296–309.
Aragón, A. M. (2013). A measure for the impact of research. Scientific Reports, 3(3), 1649–1649.
Bai, X., Zhang, J., Cui, H., Ning, Z. & Xia, F. (2016). PNCOIRank: Evaluating the impact of scholarly articles with positive and negative citations. In Proceedings of the 25th international conference companion on World Wide Web, international world wide web conferences steering committee (pp. 9–10).
Ball, P. (2006). Prestige is factored into journal ratings. Nature, 439(7078), 770–771.
Boldi, P., Santini, M. & Vigna, S. (2005). Pagerank as a function of the damping factor. In International conference on World Wide Web (pp. 557–566).
Bornmann, L., & Daniel, H.-D. (2009). The state of h index research. EMBO Reports, 10(1), 2–6.
Bornmann, L., Mutz, R., Hug, S. E., & Daniel, H.-D. (2011). A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants. Journal of Informetrics, 5(3), 346–359.
Catalini, C., Lacetera, N., & Oettl, A. (2015). The incidence and role of negative citations in science. Proceedings of the National Academy of Sciences, 112(45), 13823–13826.
Chavda, J., & Patel, A. (2016). Measuring research impact: Bibliometrics, social media, altmetrics, and the BJGP. The British Journal of General Practice, 66(642), e59–e61.
Chen, P., Xie, H., Maslov, S., & Redner, S. (2007). Finding scientific gems with Google’s PageRank algorithm. Journal of Informetrics, 1(1), 8–15.
Chen, Y.-L., & Chen, X.-H. (2011). An evolutionary pagerank approach for journal ranking with expert judgements. Journal of Information Science, 37(3), 254–272.
Cohen, J. (1977). Statistical power analysis for the behavioral sciences (pp. 19–74). New York: Academy Press.
Egghe, L. (2006). Theory and practise of the g-index. Scientometrics, 69(1), 131–152.
Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science, 178(4060), 471–479.
Garfield, E., & Sher, I. H. (1963). New factors in the evaluation of scientific literature through citation indexing. American Documentation, 14(3), 195–201.
Guimera, R., Uzzi, B., Spiro, J., & Amaral, L. A. N. (2005). Team assembly mechanisms determine collaboration network structure and team performance. Science, 308(5722), 697–702.
Harzing, A.-W., & Van Der Wal, R. (2009). A Google Scholar h-index for journals: An alternative metric to measure journal impact in economics and business. Journal of the American Society for Information Science and Technology, 60(1), 41–46.
Hirsch, J. (2010). An index to quantify an individual’s scientific research output that takes into account the effect of multiple coauthorship. Scientometrics, 85(3), 741–754.
Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National academy of Sciences of the United States of America, 102(46), 16569–16572.
Järvelin, K., & Kekäläinen, J. (2002). Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS), 20(4), 422–446.
Liu, Z., Huang, H., Wei, X. & Mao, X. (2014). Tri-rank: An authority ranking framework in heterogeneous academic networks by mutual reinforce. In 26th IEEE international conference on tools with artificial intelligence (ICTAI), IEEE (pp. 493–500).
Myers, J. L., Well, A., & Lorch, R. F. (2010). Research design and statistical analysis. London: Routledge.
Pan, R. K., & Fortunato, S. (2014). Author impact factor: Tracking the dynamics of individual scientific impact. Scientific Reports, 4, 4880–4880.
Schreiber, M. (2015). A variant of the h-index to measure recent performance. Journal of the Association for Information Science and Technology, 66(11), 2373–2380.
Shah, N. & Song, Y. (2015). S-index: Towards better metrics for quantifying research impact. arXiv preprint arXiv:1507.03650.
Simkin, M., & Roychowdhury, V. (2003). Read before you cite!. Complex Systems, 14, 269–274.
Stallings, J., Vance, E., Yang, J., Vannier, M. W., Liang, J., Pang, L., et al. (2013). Determining scientific impact using a collaboration index. Proceedings of the National Academy of Sciences, 110(24), 9680–9685.
Su, C., Pan, Y., Zhen, Y., Ma, Z., Yuan, J., Guo, H., et al. (2011). PrestigeRank: A new evaluation method for papers and journals. Journal of Informetrics, 5(1), 1–13.
Valderas, J. M. (2007). Why do team-authored papers get cited more? Science, 317(5844), 1496–1498.
van Ypersele, J.-P., et al. (2013). The maze of impact metrics. Nature, 502(7472), 423–423.
Wang, D., Song, C., & Barabási, A.-L. (2013). Quantifying long-term scientific impact. Science, 342(6154), 127–132.
Wang, Y., Tong, Y. & Zeng, M. (2013). Ranking scientific articles by exploiting citations, authors, journals, and time information. In Proceedings of the twenty-seventh AAAI conference on artificial intelligence (pp. 933–939). AAAI Press.
Wasserman, M., Zeng, X. H. T., & Amaral, L. A. N. (2015). Cross-evaluation of metrics to estimate the significance of creative works. Proceedings of the National Academy of Sciences, 112(5), 1281–1286.
Yan, E., & Ding, Y. (2011). Discovering author impact: A pagerank perspective. Information Processing & Management, 47(1), 125–134.
Yan, E., Ding, Y., & Sugimoto, C. R. (2011). P-Rank: An indicator measuring prestige in heterogeneous scholarly networks. Journal of the American Society for Information Science and Technology, 62(3), 467–477.
Zhang, C.-T. (2013). A novel triangle mapping technique to study the h-index based citation distribution. Scientific Reports, 3, 1023–1023.
Acknowledgements
The author thanks Xiaomei Bai and Haozhen Liu (School of Software, Dalian University of Technology) for support in data processing and algorithm implementations.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhang, F. Evaluating journal impact based on weighted citations. Scientometrics 113, 1155–1169 (2017). https://doi.org/10.1007/s11192-017-2510-z
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-017-2510-z