Abstract
Scientific importance ranking has long been an important research topic in scientometrics. Many indices based on citation counts have been proposed. In recent years, several graph-based ranking algorithms have been studied and claimed to be reasonable and effective. However, most current researches fall short of a concrete view of what these graph-based ranking algorithms bring to bibliometric analysis. In this paper, we make a comparative study of state-of-the-art graph-based algorithms using the APS (American Physical Society) dataset. We focus on ranking researchers. Some interesting findings are made. Firstly, simple citation-based indices like citation count can return surprisingly better results than many cutting-edge graph-based ranking algorithms. Secondly, how we define researcher importance may have tremendous impacts on ranking performance. Thirdly, some ranking methods which at the first glance are totally different have high rank correlations. Finally, the data of which time period are chosen for ranking greatly influence ranking performance but still remains open for further study. We also try to give explanations to a large part of the above findings. The results of this study open a third eye on the current research status of bibliometric analysis.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bras-Amorós, M., Domingo-Ferrer, J., & Torra, V. (2011). A bibliometric index based on the collaboration distance between cited and citing authors. Journal of Informetrics, 5(2), 248–264.
Brin, S., & Page, L. (1998). The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems, 30(1–7), 107–117.
Chen, P., Xie, H., Maslov, S., & Redner, S. (2007). Finding scientific gems with Google’s PageRank algorithms. Journal of Informetrics, 1(1), 8–15.
Das, S., Mitra, P., & Lee Giles, C. (2011). Ranking Authors in Digital Libraries. Proceedings of the 11th Annual International ACM/IEEE Joint Conference on Digital Libraries, pp. 251–254.
Ding, Y., Yan, E., Frazho, R., & Caverlee, J. (2009). PageRank for Ranking Authors in Co-citation Networks. Journal of the American Society of Information Science and Technology, 60(11), 2229–2243.
Egghe, L. (2006). Theory and practice of the g-index. Scientometrics, 69(1), 131–152.
Eom, Y.-H., & Fortunato, S. (2011). Characterizing and modeling citation dynamics. PLoS ONE, 6(9), 1–7.
Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science, 178(60), 471–479.
Hajra, K. B., & Sen, P. (2004). Aging in citation networks. Physica A: Statistical Mechanics and its Applications, 346(1–2), 44–48.
Hajra, K. B., & Sen, P. (2006). Modelling aging characteristics in citation networks. Physica A: Statistical Mechanics and its Applications, 368(2), 575–582.
Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences, 102(46), 4–16569.
Jiang, X., Sun, X., & Zhuge, H. (2012). Towards an effective and unbiased ranking of scientific literature through mutual reinforcements. Proceedings of the 21st ACM Conference on Information and Knowledge Management.
Kleinberg, J. M. (1999). Authoritative sources in a hyperlinked environment. Journal of the ACM, 46(5), 604–632.
Lefebvre, M. (2006). Applied stochastic processes. New York: Springer.
Lempel, R., & Moran, S. (2001). SALSA: the stochastic approach for link-structure analysis. ACM Transactions on Internet Technology, 19(2), 131–169.
Li, X., Liu, B., & Yu, P. (2008). Time sensitive ranking with application to publication search. Proceedings of the ninth IEEE International Conference on Data Mining, pp. 893-898.
Nerur, S., Sikora, R., Mangalaraj, G., & Balijepally, V. (2005). Assessing the relative influence of journals in a citation network. Communications of the ACM, 48(11), 71–74.
Ng, A. Y., Zheng, A. X., & Jordan, M. I. (2001). Stable algorithms for link analysis. Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 258–266.
Radicchi, F., Fortunato, S., Markines, B., & Vespignani, A. (2009). Diffusion of scientific credits and the ranking of scientists. Physical Review E, 80(5), 056103–056112.
Sayyadi, H., & Getoor, L. (2009). FutureRank: ranking scientific articles by predicting their future PageRank. Proceedings of 2009 SIAM Conference on Data Mining, pp. 533–544.
Silagadze, Z. (2010). Citation entropy and research impact estimation. Acta Physica Polonica A, B41, 2325–2333.
Walker, D., Xie, H., Yan, K–. K., & Maslov, S. (2007). Ranking scientific publications using a model of network traffic. Journal of Statistical Mechanics, 7, 06010–06019.
Wang, M., Yu, G., & Yu, D. (2009). Effect of the age of papers on the preferential attachment in citation networks. Physica A: Statistical Mechanics and its Applications, 368(2), 575–582.
Yan, E., & Ding, Y. (2009). Applying centrality measures to impact analysis: a co-authorship network analysis. Journal of the American Society of Information Science and Technology, 60(10), 2107–2118.
Yan, E., Ding, Y., & Sugimoto, C. R. (2011). P-Rank: an indicator measuring prestige in heterogeneous scholarly networks. Journal of the American Society of Information Science and Technology, 62(3), 467–477.
Zhou, D., Orshanskiy, S. A., Zha, H., & Lee Giles, C. (2007). Co-Ranking authors and documents in a heterogeneous network. Proceedings of the Seventh IEEE International Conference on Data Mining, pp. 739–744.
Zhuge, H., & Zhang, J. (2010). Topological centrality and its e-science applications. Journal of the American Society of Information Science and Technology, 61(9), 1824–1841.
Acknowledgments
This work is supported by National Science Foundation of China (61075074 and 61070183), Natural Science Foundation of Chongqing (No.cstc2012jjB40012), and the Key Discipline Fund of National 211 Project (Southwest University: NSKD11013).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Jiang, X., Sun, X. & Zhuge, H. Graph-based algorithms for ranking researchers: not all swans are white!. Scientometrics 96, 743–759 (2013). https://doi.org/10.1007/s11192-012-0943-y
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-012-0943-y