An Internet measure of the value of citations
Introduction
Currently, the impact of a scientific publication is often measured by the number of citations it receives. Perhaps we are suffering from an over-analysis of citations for the purposes of assessing scientists and universities productivity, impact, or prestige—the examination of citations of scientific publications has become a cottage industry in higher education. This approach has been taken to extremes both for the assessment of individuals and as a measure of the productivity and influence of entire universities or even academic systems. Pioneered in the 1950s in the United States, bibliometrics was invented as a tool for tracing research ideas, the progress of science and the impact of scientific work. First developed for the “hard” sciences, it was later expanded to include the social sciences and humanities.
The citation system was invented mainly as a way to understand how scientific discoveries and innovations are communicated and how research functions [1]. It was not initially seen as a tool for evaluating individual scientists, entire universities or academic systems. Hence, the citation system is useful for tracking how scientific ideas are propagated among researchers and how individual scientists use and communicate research findings. The use of citation analysis for the assessment of research productivity or impact questionably extends the original reasons for creating the bibliometric system. Evaluators and rankers need to go back to the drawing board in considering a reliable system for the accurate measurement of the scientific and scholarly work of individuals and institutions. The unwieldy and inappropriate use of citation analysis and bibliometrics for the evaluation and ranking of research and researchers does not serve higher education well and it entrenches existing inequalities.
More recently, a new index based on citations, the h-index, has been proposed as an indicator of overall productivity and impact of the published work of a researcher [15]. The h-index of a researcher is the largest integer h such that at least h publications by this researcher have no less than h citations each. For example, an author with an h-index of 20 must have at most 20 publications with 21 or more citations and at least 20 publications with 20 citations each.1 This index can easily be determined from the “times cited” in the Thomson ISI Web of Science or Google Scholar and it provides a metric for the author’s productivity in terms of citations.
The h-index focuses more on measuring productivity than on measuring the impact and influence of the dissemination of a publication. However, some h-index variations attempt to capture the latter [4], [5]. Measuring impact by the number of new authors who cite a publication appears to be a more accurate measure than measuring it by the h-index because it reflects the utility of an author’s work to various individuals rather than only the same people. Thus, any type of direct or indirect self-citations should be discounted to a certain degree. Moreover, if impact signifies the importance of knowledge dissemination in publications citing the given publication, then citing a publication with a greater impact should in turn endow a higher impact to the cited publication.
In this work, we propose a new approach for measuring the impact of publications and compare it with an author ranking computed using the PageRank algorithm [17]. To the best of our knowledge, PageRank was originally inspired by the scientific bibliometric system (citations), but only recently has it been applied to measure the impact of journals, publications and scientists. The success of Google’s method of ranking web pages has inspired numerous measures of journal impact that apply social network analysis to citation networks. Pinski and Narin [20] (predating PageRank) proposed ranking journals according to their eigenvector centrality in a citation network. Extending this idea, we propose a more accurate measure of impact than those based on the h-index. Our measurement is based not on a row citation count but on the impact of the citing publications and their distance from self-citations. Section 2 provides a precise explanation of our approach, introducing scientific currency tokens as a measure of the impact of citations. Section 3 presents an algorithm for estimating this value from a network of publications and authors connected by citations. Section 4 presents an example of how many tokens would be assigned to each citation in a network of nine citations among 6 publications by four authors. Section 5 describes an application of the PageRank algorithm to the same example followed by a comparison of the values of the citations calculated by both algorithms. Section 6 describes a method to compute the citation earnings of each author when there are multiple authors for a publication and shows an example of how to apply the h-index to CENTs instead of to citations. The conclusions and prospective future work are provided in 7.
Section snippets
CENTs – scientific currency tokens
We first describe the heuristics behind our model. We advocate measuring the value of each citation in sCientific currENcy Tokens (CENTs). The introduction of this currency was inspired by complementary currencies for the scientific communities proposed in [8], [11], [12] and also conceptualized as tokens or measure of reputation by [10], [18]. Scientists are assumed to hold a new scientific currency and to have rational expectations with it [7]. The initial value of a publication is one CENT
Raw citations and acknowledged citations
In our approach, citations to a publication are distributed among the authors of the publication. This is simple in the case of single-author publications, as the value of the publication is passed onto its author. A straightforward extension to multi-author publications, which we use here, is to divide the value of each publication equally among its authors. A more sophisticated extension could allow the authors to decide among themselves how the credit is divided among them and such a
Example
In this section, we consider the following citation pattern of a sequence of publications and their interrelated citations shown in Fig. 2.
The citation pattern is represented by the transition matrix D of graph GP, which is shown in Table 2.
This matrix corresponds to the graph of citations shown in Fig. 3:
The functions b and s, computed according to Eqs. (2), (5), respectively, are shown in Table 3.
Applying Eq. (6), the acknowledged citation matrix a(k, l) shown in Table 4 is generated:
The
An Internet measure of the citation value
As said in Section 1, PageRank [17] can also be used to estimate the value of citations. The success of Google’s method of ranking web pages has inspired numerous measures of journal impact that apply social network analysis to citation networks. Pinski et al. [20] (predating PageRank) proposed a journal ranking based on their eigenvector centrality in a citation network. They suggested the use of a recursive impact factor to give citations from high-impact journals greater weight than
Multiple authors
The model with unique authors presented earlier can easily be generalized to publications with multiple authors. Currently, a citation of a multi-authored publication implies a multiplication of this citation by the number of authors of the cited publication. If a publication Pi has ki authors, one inbound raw citation ri to the publication generates ki inbound raw citations. Coauthors share the ownership of a publication, and historically they are assumed to have equal shares. In reality,
Final discussion
This is an approach for measuring the impact of authors by calculating first (with some heuristics) the estimated value of acknowledged citations and then converting this value into CENTs as a measure of the value of a publication. The value of publications in CENTs is propagated through the outgoing citations of a publication. We compared the resulting rankings of authors generated by our approach with that of PageRank and two citation-based rankings. More accurate algorithms for acknowledged
Acknowledgments
This research was funded by the European Union Project No. 238887, A unique European citizens’ attention service (iSAC6+) IST-PSP, the ACC1Ó grant ASKS – Agents for Social Knowledge Search – Catalan Government, the Spanish MCI project TIN2010-17903. Comparative approaches to the implementation of intelligent agents in digital preservation from a perspective of the automation of social networks, 2009 BE-1-00229 of the AGAUR awarded to Josep Lluis de la Rosa, and the CSI-ref.2009SGR-1202.
References (23)
- et al.
Finding scientific gems with google’s pagerank algorithm
Journal of Informetrics
(2007) - et al.
Refining dermatology journal impact factors using pagerank
Journal of the American Academy Dermatology
(2007) A computer-aided bibliometric system to generate core article ranked lists in interdisciplinary subjects
Information Sciences
(2007)- et al.
On the Floyd–Warshall algorithm for logic programs
The Journal of Logic Programming
(1999) - et al.
Citation influence for journal aggregates of scientific publications: theory with application to literature of physics
Information Processing and Management
(1976) - et al.
A non-linear index to evaluate a journal’s scientific impact
Information Sciences
(2010) The Tyranny of Citations
(2006)- et al.
Journal status
Scientometrics
(2006) - et al.
What do we know about the h index?
Journal of American Society Information Science and Technology
(2007) - et al.
Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine
Journal of the American Society for Information Science and Technology
(2008)
Are there really two types of h index variants? A validation study by using molecular life sciences data
Cited by (9)
Social networks and open innovation: Business academic productivity
2021, Journal of Open Innovation: Technology, Market, and ComplexityA fast method for identifying worldwide scientific collaborations using the Scopus database
2018, Telematics and InformaticsCitation Excerpt :Journal metrics are often based on the journal impact factor, which is measured using Thomson’s Journal Citation Reports (JCR), Scimago Journal & Country Rank (SJR), etc. Among the author metrics, the total number of citations (Szymanski et al., 2012), as well as the h-index (Hirsch, 2005), or variations of it (Bornmann et al., 2008; Papavlasopoulos et al., 2010) stand out. However, these metrics do not take into account other aspects that are of great importance, such as the structure of collaborative relationships between researchers.
Scopus analysis of the academic research performed by public Universities in galicia and North of Portugal
2020, Information Resources Management JournalMetrics for openness
2017, Journal of the Association for Information Science and TechnologyAssessing the management of electronic scientific journals using hybrid FDELPHI and FAHP methodology
2017, Fuzzy Analytic Hierarchy Process