Abstract
Scientific impact evaluation is a long-standing problem in scientometrics. Graph-ranking methods are often employed to account for the collective diffusion process of scientific credit among researchers or their publications. One key issue, however, is still up in the air: what is the appropriate level for scientific credit diffusion, researcher level or paper level? In this paper, we tackle this problem via an anatomy of the credit diffusion mechanism underlying both researcher level and paper level graph-ranking methods. We find that researcher level and paper level credit diffusions are actually two aggregations of a fine-grained authorship level credit diffusion. We further find that researcher level graph-ranking methods may cause misallocation of scientific credit, but paper level graph-ranking methods do not. Consequently, researcher level methods often fail to identify researchers with high quality but low productivity. This finding indicates that scientific credit is fundamentally derived from “paper citing paper” rather than “researcher citing researcher”. We empirically verify our findings using American Physical Review publication dataset spanning over one century.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Chen, P., Xie, H., Maslov, S., & Redner, S. (2007). Finding scientific gems with google’s pagerank algorithm. Journal of Informetrics, 1(1), 8–15.
Ding, Y. (2011). Applying weighted pagerank to author citation networks. Journal of the American Society for Information Science and Technology, 62(2), 236–245.
Egghe, L. (2006). Theory and practise of the g-index. Scientometrics, 69(1), 131–152.
Fortin, J. M., & Currie, D. J. (2013). Big science vs. little science: How scientific impact scales with funding. PLoS One, 8(6), e65263.
Garfield, E. (1972). Citation analysis as a tool in journal evaluation–Journals can be ranked by frequency and impact of citations for science policy studies. Science, 178(4060), 471–479.
Giles, C. L., & Councill, I. G. (2004). Who gets acknowledged: Measuring scientific contributions through automatic acknowledgment indexing. Proceedings of the National Academy of Sciences of the United States of America, 101(51), 17599–17604.
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). The Leiden manifesto for research metrics. Nature, 520(7548), 429–431.
Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.
Jiang, X. R., Sun, X. P., & Zhuge, H. (2013). Graph-based algorithms for ranking researchers: Not all swans are white!. Scientometrics, 96(3), 743–759.
Kim, J., & Diesner, J. (2015). Distortive effects of initial-based name disambiguation on measurements of large-scale coauthorship networks. Journal of the Association for Information Science and Technology. doi:10.1002/asi.23489.
Kim, J., & Kim, J. (2015). Rethinking the comparison of coauthorship credit allocation schemes. Journal of Informetrics, 9(3), 667–673.
Klosik, D. F., & Bornholdt, S. (2014). The citation wake of publications detects nobel laureates’ papers. PLoS One, 9(12), e113184.
Kreiman, G., & Maunsell, J. H. R. (2011). Nine criteria for a measure of scientific output. Frontiers in Computational Neuroscience, 5, 48.
Martin, T., Ball, B., Karrer, B., & Newman, M. E. J. (2013). Coauthorship and citation patterns in the physical review. Physical Review E, 88(1), 012814.
Newman, M. E. J. (2004). Coauthorship networks and patterns of scientific collaboration. Proceedings of the National Academy of Sciences of the United States of America, 101(suppl 1), 5200–5205.
Neylon, C., & Wu, S. (2009). Article-level metrics and the evolution of scientific impact. PLoS Biology, 7(11), 2459.
Petersen, A. M. (2015). Quantifying the impact of weak, strong, and super ties in scientific careers. Proceedings of the National Academy of Sciences of the United States of America, 112(34), E4671–E4680.
Petersen, A. M., Wang, F. Z., & Stanley, J. E. (2010). Methods for measuring the citations and productivity of scientists across time and discipline. Physical Review E, 81(3), 036114.
Radicchi, F., Fortunato, S., Markines, B., & Vespignani, A. (2009). Diffusion of scientific credits and the ranking of scientists. Physical Review E, 80(5), 056103.
Senanayake, U., Piraveenan, M., & Zomaya, A. (2015). The pagerank-index: Going beyond citation counts in quantifying scientific impact of researchers. PloS One, 10(8), e0134794.
Shen, H. W., & Barabási, A. L. (2014). Collective credit allocation in science. Proceedings of the National Academy of Sciences of the United States of America, 111(34), 12325–12330.
Shen, H. W., Wang, D. S., Song, C. M., & Barabási, A. L. (2014). Modeling and predicting popularity dynamics via reinforced poisson processes. In Twenty-eighth AAAI conference on artificial intelligence (pp. 291–297).
Stallings, J., Vance, E., Yang, J. S., Vannier, M. W., Liang, J. M., Pang, L. J., et al. (2013). Determining scientific impact using a collaboration index. Proceedings of the National Academy of Sciences of the United States of America, 110(24), 9680–9685.
Strotmann, A., & Zhao, D. Z. (2012). Author name disambiguation: What difference does it make in author-based citation analysis? Journal of the American Society for Information Science and Technology, 63(9), 1820–1833.
Vinkler, P. (2009). The \(\pi\)-index: A new indicator for assessing scientific impact. Journal of Information Science, 35(5), 602–612.
Wu, Q. (2010). The w-index: A measure to assess scientific impact by focusing on widely cited papers. Journal of the American Society for Information Science and Technology, 61(3), 609–614.
Wuchty, S., Jones, B. F., & Uzzi, B. (2007). The increasing dominance of teams in production of knowledge. Science, 316(5827), 1036–1039.
Yao, L. Y., Wei, T., Zeng, A., Fan, Y., & Di, Z. R. (2014). Ranking scientific publications: The effect of nonlinearity. Scientific Reports, 4, 6663.
Acknowledgments
This work was funded by the National Basic Research Program of China (973 Program) under Grant Number 2014CB340401, the National High-tech R&D Program of China (863 Program) under Grant Number 2014AA015103, and the National Natural Science Foundation of China under Grant Numbers 61472400, 61425016, 61433014. H. W. Shen is also funded by Youth Innovation Promotion Association CAS.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wang, H., Shen, HW. & Cheng, XQ. Scientific credit diffusion: Researcher level or paper level?. Scientometrics 109, 827–837 (2016). https://doi.org/10.1007/s11192-016-2057-4
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-016-2057-4