Skip to main content
Springer Nature Link
Log in

Two citation-based indicators to measure latent referential value of papers

  • Published:
Scientometrics Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

Traditional bibliometric indicators aim at helping academic administrators or research investors measure the influence of publications. These indicators focus on how to quantify and compare the scientific output of researchers. However, little attention has been paid to the aspect that bibliometric indicators can also be used to help scientists find valuable referential papers. In this paper, we propose three points to characterize valuable referential papers: first, valuable referential papers always are high-quality research; second, valuable referential papers are closely related to a considerable quantity of recent papers; third, valuable referential papers lead to hotspots which attract papers to follow successively. We extract the critical subnetwork from the original citation network which only reserves the significant nodes and edges that meet the three preceding points. Then we present two indicators on the basis of the critical subnetwork. The experimental results demonstrate that papers recommended by our indicators are relatively new and our indicators have greater Spearman’s rank correlation coefficients with the future citation count compared with other bibliometric indicators like the raw citation count.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Abramo, G., Costa, C., & D’Angelo, C. A. (2015). A multivariate stochastic model to assess research performance. Scientometrics, 102(2), 1755–1772. doi:10.1007/s11192-014-1474-5.

    Article  Google Scholar 

  • Adam, D. (2002). Citation analysis: The counting house. Nature, 415(6873), 726–729.

    Article  Google Scholar 

  • Adie, E., & Roe, W. (2013). Altmetric: Enriching scholarly content with article-level discussion and metrics. Learned Publishing, 26(1), 11. doi:10.1087/20130103.

    Article  Google Scholar 

  • Beel, J., & Gipp, B. Google Scholar’s ranking algorithm: The impact of citation counts (an empirical study). In Research challenges in information science, 2009. RCIS 2009. Third international conference on IEEE, 2009 (pp. 439–446).

  • Braun, T., Glanzel, W., & Schubert, A. (2006). A Hirsch-type index for journals. Scientometrics, 69(1), 169–173. doi:10.1007/s11192-006-0147-4.

    Article  Google Scholar 

  • Chen, P., Xie, H., Maslov, S., & Redner, S. (2007). Finding scientific gems with Google’s PageRank algorithm. Journal of Informetrics, 1(1), 8–15. doi:10.1016/j.joi.2006.06.001.

    Article  Google Scholar 

  • Davis, P. M. (2008). Eigenfactor: Does the principle of repeated improvement result in better estimates than raw citation counts? Journal of the American Society for Information Science and Technology, 59(13), 2186–2188. doi:10.1002/asi.20943.

    Article  Google Scholar 

  • Egghe, L. (2006). Theory and practise of the g-index. Scientometrics, 69(1), 131–152. doi:10.1007/s11192-006-0144-7.

    Article  MathSciNet  Google Scholar 

  • Franceschini, F., Galetto, M., Maisano, D., & Mastrogiacomo, L. (2012). The success-index: An alternative approach to the h-index for evaluating an individual’s research output. Scientometrics, 92(3), 621–641. doi:10.1007/s11192-011-0570-z.

    Article  Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572. doi:10.1073/pnas.0507655102.

    Article  Google Scholar 

  • Hirsch, J. E. (2007). Does the h index have predictive power? Proceedings of the National Academy of Sciences of the United States of America, 104(49), 19193–19198. doi:10.1073/pnas.0707962104.

    Article  Google Scholar 

  • Ho, Y. S. (2004). Citation review of Lagergren kinetic rate equation on adsorption reactions. Scientometrics, 59(1), 171–177.

    Article  Google Scholar 

  • Ibanez, A., Larranaga, P., & Bielza, C. (2009). Predicting citation count of bioinformatics papers within 4 years of publication. Bioinformatics, 25(24), 3303–3309. doi:10.1093/bioinformatics/btp585.

    Article  Google Scholar 

  • Jin, B. H., Liang, L. M., Rousseau, R., & Egghe, L. (2007). The R- and AR-indices: Complementing the h-index. Chinese Science Bulletin, 52(6), 855–863. doi:10.1007/s11434-007-0145-9.

    Article  Google Scholar 

  • Lundberg, J. (2007). Lifting the crown-citation z-score. Journal of Informetrics, 1(2), 145–154. doi:10.1016/j.joi.2006.09.007.

    Article  Google Scholar 

  • MacRoberts, M. H., & MacRoberts, B. R. (1996). Problems of citation analysis. Scientometrics, 36(3), 435–444. doi:10.1007/Bf02129604.

    Article  Google Scholar 

  • MacRoberts, M. H., & MacRoberts, B. R. (2010). Problems of citation analysis: A study of uncited and seldom-cited influences. Journal of the American Society for Information Science and Technology, 61(1), 1–12. doi:10.1002/asi.21228.

    Article  Google Scholar 

  • Price, D. J. D. (1965). Networks of scientific papers. Science, 149(3683), 510.

    Article  Google Scholar 

  • Schubert, A., & Braun, T. (1986). Relative indicators and relational charts for comparative-assessment of publication output and citation impact. Scientometrics, 9(5–6), 281–291. doi:10.1007/Bf02017249.

    Article  Google Scholar 

  • Seglen, P. O. (1997). Citations and journal impact factors: Questionable indicators of research quality. Allergy, 52(11), 1050–1056.

    Article  Google Scholar 

  • Senanayake, U., Piraveenan, M., & Zomaya, A. (2015). The pagerank-index: Going beyond citation counts in quantifying scientific impact of researchers. PLoS One, 10(8), 222–229. doi:10.1371/journal.pone.0134794.

    Article  Google Scholar 

  • Shibata, N., Kajikawa, Y., & Sakata, I. (2012). Link prediction in citation networks. Journal of the American Society for Information Science and Technology, 63(1), 78–85.

    Article  Google Scholar 

  • Thorne, F. C. (1977). Citation index—another case of spurious validity. Journal of Clinical Psychology, 33(4), 1157–1161. doi:10.1002/1097-4679(197710)33:4<1157:Aid-Jclp2270330453>3.0.Co;2-B.

    Article  MathSciNet  Google Scholar 

  • Van Raan, A. F. J. (1996). Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises. Scientometrics, 36(3), 397–420.

    Article  Google Scholar 

  • Van Raan, A. F. J. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics, 67(3), 491–502. doi:10.1556/Scient.67.2006.3.10.

    Article  Google Scholar 

  • Waltman, L., Costas, R., & Jan van Eck, N. (2012). Some limitations of the h index: A commentary on Ruscio and colleagues’ analysis of bibliometric indices. Measurement: Interdisciplinary Research and Perspectives, 10(3), 172–175.

    Google Scholar 

  • Waltman, L., van Eck, N. J., van Leeuwen, T. N., Visser, M. S., & van Raan, A. F. J. (2011). Towards a new crown indicator: Some theoretical considerations. Journal of Informetrics, 5(1), 37–47. doi:10.1016/j.joi.2010.08.001.

    Article  Google Scholar 

Download references

Acknowledgments

This work was jointly supported by the National Natural Science Foundation of China (Grant nos. 61173111 and 60774086) and the Ph.D. Programs Foundation of Ministry of Education of China (Grant no. 20090201110027).

Author information

Authors and Affiliations

  1. Systems Engineering Institute, Xi’an Jiaotong University, Xi’an, 710049, China

    Zhi Li, Qinke Peng & Che Liu

Authors
  1. Zhi Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Qinke Peng
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Che Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Qinke Peng.

Ethics declarations

Conflict of interest

None.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Z., Peng, Q. & Liu, C. Two citation-based indicators to measure latent referential value of papers. Scientometrics 108, 1299–1313 (2016). https://doi.org/10.1007/s11192-016-2000-8

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-016-2000-8

Keywords