Elsevier

Information Sciences

Volume 185, Issue 1, 15 February 2012, Pages 18-31
Information Sciences

An Internet measure of the value of citations

https://doi.org/10.1016/j.ins.2011.08.005Get rights and content

Abstract

A new method for computing the value of citations is introduced and compared with the PageRank algorithm for author ranking. In our proposed approach, the value of each publication is expressed in CENTs (sCientific currENcy Tokens). The publication’s value is then divided by the number of citations made by that publication to yield a value for each citation. As citations are the acknowledgements of the work by authors other than oneself (indicating that it has been useful), self-citations count as zero in acknowledged citation value. Circular citations, a generalized type of self-citation, are considered to have a reduced acknowledged citation value. Finally, we propose a modification of the h-index to define it as the largest integer such that the i-th publication (on the list of publications sorted by their value in CENTs) is worth more than i CENTs. This new index, termed the i-index or i2 in short, appears to be a more precise measure of the impact of publications and their authors’ productivity than the h-index.

Introduction

Currently, the impact of a scientific publication is often measured by the number of citations it receives. Perhaps we are suffering from an over-analysis of citations for the purposes of assessing scientists and universities productivity, impact, or prestige—the examination of citations of scientific publications has become a cottage industry in higher education. This approach has been taken to extremes both for the assessment of individuals and as a measure of the productivity and influence of entire universities or even academic systems. Pioneered in the 1950s in the United States, bibliometrics was invented as a tool for tracing research ideas, the progress of science and the impact of scientific work. First developed for the “hard” sciences, it was later expanded to include the social sciences and humanities.

The citation system was invented mainly as a way to understand how scientific discoveries and innovations are communicated and how research functions [1]. It was not initially seen as a tool for evaluating individual scientists, entire universities or academic systems. Hence, the citation system is useful for tracking how scientific ideas are propagated among researchers and how individual scientists use and communicate research findings. The use of citation analysis for the assessment of research productivity or impact questionably extends the original reasons for creating the bibliometric system. Evaluators and rankers need to go back to the drawing board in considering a reliable system for the accurate measurement of the scientific and scholarly work of individuals and institutions. The unwieldy and inappropriate use of citation analysis and bibliometrics for the evaluation and ranking of research and researchers does not serve higher education well and it entrenches existing inequalities.

More recently, a new index based on citations, the h-index, has been proposed as an indicator of overall productivity and impact of the published work of a researcher [15]. The h-index of a researcher is the largest integer h such that at least h publications by this researcher have no less than h citations each. For example, an author with an h-index of 20 must have at most 20 publications with 21 or more citations and at least 20 publications with 20 citations each.1 This index can easily be determined from the “times cited” in the Thomson ISI Web of Science or Google Scholar and it provides a metric for the author’s productivity in terms of citations.

The h-index focuses more on measuring productivity than on measuring the impact and influence of the dissemination of a publication. However, some h-index variations attempt to capture the latter [4], [5]. Measuring impact by the number of new authors who cite a publication appears to be a more accurate measure than measuring it by the h-index because it reflects the utility of an author’s work to various individuals rather than only the same people. Thus, any type of direct or indirect self-citations should be discounted to a certain degree. Moreover, if impact signifies the importance of knowledge dissemination in publications citing the given publication, then citing a publication with a greater impact should in turn endow a higher impact to the cited publication.

In this work, we propose a new approach for measuring the impact of publications and compare it with an author ranking computed using the PageRank algorithm [17]. To the best of our knowledge, PageRank was originally inspired by the scientific bibliometric system (citations), but only recently has it been applied to measure the impact of journals, publications and scientists. The success of Google’s method of ranking web pages has inspired numerous measures of journal impact that apply social network analysis to citation networks. Pinski and Narin [20] (predating PageRank) proposed ranking journals according to their eigenvector centrality in a citation network. Extending this idea, we propose a more accurate measure of impact than those based on the h-index. Our measurement is based not on a row citation count but on the impact of the citing publications and their distance from self-citations. Section 2 provides a precise explanation of our approach, introducing scientific currency tokens as a measure of the impact of citations. Section 3 presents an algorithm for estimating this value from a network of publications and authors connected by citations. Section 4 presents an example of how many tokens would be assigned to each citation in a network of nine citations among 6 publications by four authors. Section 5 describes an application of the PageRank algorithm to the same example followed by a comparison of the values of the citations calculated by both algorithms. Section 6 describes a method to compute the citation earnings of each author when there are multiple authors for a publication and shows an example of how to apply the h-index to CENTs instead of to citations. The conclusions and prospective future work are provided in 7.

Section snippets

CENTs – scientific currency tokens

We first describe the heuristics behind our model. We advocate measuring the value of each citation in sCientific currENcy Tokens (CENTs). The introduction of this currency was inspired by complementary currencies for the scientific communities proposed in [8], [11], [12] and also conceptualized as tokens or measure of reputation by [10], [18]. Scientists are assumed to hold a new scientific currency and to have rational expectations with it [7]. The initial value of a publication is one CENT

Raw citations and acknowledged citations

In our approach, citations to a publication are distributed among the authors of the publication. This is simple in the case of single-author publications, as the value of the publication is passed onto its author. A straightforward extension to multi-author publications, which we use here, is to divide the value of each publication equally among its authors. A more sophisticated extension could allow the authors to decide among themselves how the credit is divided among them and such a

Example

In this section, we consider the following citation pattern of a sequence of publications and their interrelated citations shown in Fig. 2.

The citation pattern is represented by the transition matrix D of graph GP, which is shown in Table 2.

This matrix corresponds to the graph of citations shown in Fig. 3:

The functions b and s, computed according to Eqs. (2), (5), respectively, are shown in Table 3.

Applying Eq. (6), the acknowledged citation matrix a(k, l) shown in Table 4 is generated:

The

An Internet measure of the citation value

As said in Section 1, PageRank [17] can also be used to estimate the value of citations. The success of Google’s method of ranking web pages has inspired numerous measures of journal impact that apply social network analysis to citation networks. Pinski et al. [20] (predating PageRank) proposed a journal ranking based on their eigenvector centrality in a citation network. They suggested the use of a recursive impact factor to give citations from high-impact journals greater weight than

Multiple authors

The model with unique authors presented earlier can easily be generalized to publications with multiple authors. Currently, a citation of a multi-authored publication implies a multiplication of this citation by the number of authors of the cited publication. If a publication Pi has ki authors, one inbound raw citation ri to the publication generates ki inbound raw citations. Coauthors share the ownership of a publication, and historically they are assumed to have equal shares. In reality,

Final discussion

This is an approach for measuring the impact of authors by calculating first (with some heuristics) the estimated value of acknowledged citations and then converting this value into CENTs as a measure of the value of a publication. The value of publications in CENTs is propagated through the outgoing citations of a publication. We compared the resulting rankings of authors generated by our approach with that of PageRank and two citation-based rankings. More accurate algorithms for acknowledged

Acknowledgments

This research was funded by the European Union Project No. 238887, A unique European citizensattention service (iSAC6+) IST-PSP, the ACC1Ó grant ASKSAgents for Social Knowledge Search – Catalan Government, the Spanish MCI project TIN2010-17903. Comparative approaches to the implementation of intelligent agents in digital preservation from a perspective of the automation of social networks, 2009 BE-1-00229 of the AGAUR awarded to Josep Lluis de la Rosa, and the CSI-ref.2009SGR-1202.

References (23)

  • L. Bornmann et al.

    Are there really two types of h index variants? A validation study by using molecular life sciences data

  • Cited by (9)

    • Social networks and open innovation: Business academic productivity

      2021, Journal of Open Innovation: Technology, Market, and Complexity
    • A fast method for identifying worldwide scientific collaborations using the Scopus database

      2018, Telematics and Informatics
      Citation Excerpt :

      Journal metrics are often based on the journal impact factor, which is measured using Thomson’s Journal Citation Reports (JCR), Scimago Journal & Country Rank (SJR), etc. Among the author metrics, the total number of citations (Szymanski et al., 2012), as well as the h-index (Hirsch, 2005), or variations of it (Bornmann et al., 2008; Papavlasopoulos et al., 2010) stand out. However, these metrics do not take into account other aspects that are of great importance, such as the structure of collaborative relationships between researchers.

    • Metrics for openness

      2017, Journal of the Association for Information Science and Technology
    View all citing articles on Scopus
    View full text