Compressing multiple scales of impact detection by Reference Publication Year Spectroscopy

Highlights

• RPYS provides a citation-based technique for exploring historically important papers for a given research field.

• Up until this point, there has been no standard procedure for comparing results across multiple RPYS analyses.

• We present a method allowing researchers to analyze and visualize the results of multiple RPYS procedures simultaneously.

Abstract

Reference Publication Year Spectroscopy (RPYS) is a scientometric technique that effectively reveals punctuated peaks of historical scientific impact on a specified research field or technology. In many cases, a seminal discovery serves as the driving force underlying any given peak. Importantly, the results from a RPYS analyses are represented on their own distinct scales, the bounds of which vary considerably across analyses. This makes comparing years of punctuated impact across multiple RPYS analyses problematic. In this paper, we propose a data transformation and visualization technique that resolves this challenge. Specifically, using a rank-order normalization procedure, we compress the results of multiple RPYS analyses into a single, consistent rank scale that clearly highlights years of punctuated impact across RPYS analyses. We suggest that rank transformation increases the effectiveness of this scientometric technique to reveal the scope of historical impact of seminal works by allowing researchers to simultaneously consider results from multiple RPYS analyses.

Introduction

Citations signify the relevance of prior research or invention. In the scientific community, the aggregation of citations attributed to a specific work is commonly taken as a central indicator of its scholarly impact (De Solla Price, 1965, Garfield et al., 1978, Radicchi et al., 2008). More generally, citations and citation counts are thought to represent how knowledge accumulates, combines and transfers to generate new ideas and discoveries. Since citations function as linkages between scientific works, citation records provide an opportunity to quantitatively identify seminal contributions to a given research field or technology (Kostoff and Shlesinger, 2005, Marx et al., 2014, van Raan, 2000).

One technique leveraging citations to detect important scientific contributions is Reference Publication Year Spectroscopy (RPYS). RPYS offers a quantitative approach to assist in identifying the historical roots of research fields and topics (Marx et al., 2014). To accomplish this, RPYS considers the references cited by a cohort of publications resulting from a particular database query. By way of example, consider a search query for topic X that returns only one article. If this article, published in 1990, cites a reference published in 1980, then 1980 is used as a data point in the foregoing analysis. As such, after a set of publications is returned from a database query, the publication date of each cited reference from this set of publications is extracted and mapped onto a frequency distribution sorted by time. The resulting visualization often reveals punctuated peaks in the distribution. These peaks correspond to years containing a larger number of cited references within discrete bins of time. Often, these peaks are driven by a large number of references to a seminal work in the field. To date, RPYS has been successfully applied to investigations of important early contributions in several research topics (Leydesdorff et al., 2014, Marx et al., 2014, Marx and Bornmann, 2013, Wray and Bornmann, 2014).

There is, however, a major challenge with the current methodology. Namely, the results produced by a given RPYS analysis are represented within their own distinct range or scale, the bounds of which vary considerably across analyses. In other words, using the presently defined RPYS technique, it could be difficult to compare patterns of maxima for the cited references of a small research fields with those of the cited references from a much larger research fields. Making RPYS analyses amenable to large-scale comparative analysis is an important extension of the technique for future applications. For instance, it would allow analysts to more readily evaluate whether a large number of research topics show a similar history of important findings as demonstrated through cited works. A second possibility is that being able to estimate the similarities between the citation histories for various subfields might open-up an entirely novel venue for defining the relationships between these subfields – with the assumption being that research areas correspondingly informed by the same seminal works are more similar than those that are not.

To address this shortcoming, we demonstrate here how the addition of a simple data transformation procedure to the standard RPYS methodology can aid in the detection of shared patterns of maxima for the cited references across RPYS analyses, which potentially suggest common historical influences. Specifically, we adopt the use of a rank-transformation procedure commonly used in inferential statistics to perform non-parametric analyses (Conover and Iman, 1981, Labovitz, 1970). This transformation compresses the multiple scales produced from various RPYS analyses into a single rank scale that allows researchers to identify years of punctuated impact across RPYS analyses. We describe a visualization procedure that efficiently represents data from multiple RPYS analyses concurrently.

To demonstrate the efficacy of this procedure, we begin with a publication that we suspect a priori has meaningfully impacted numerous research topics: the Viterbi algorithm first published by Andrew Viterbi in 1967. In this groundbreaking work, Viterbi describes an algorithm that identifies the most likely sequence of hidden states associated with a sequence of known or observed states. The algorithm is widely used in stochastic models and error-correcting (or decoding convolutional codes) as well as in a variety of computational procedures pertaining to machine speech recognition (Viterbi, 2006). Given this, we performed six RPYS analyses for research topics pulled from the Web of Science that all pertained to the development or use of the Viterbi algorithm (Viterbi, 1967). The Viterbi algorithm's impact and use in a wide array of research communities, from statisticians studying stochastic models to engineers and computer scientists working on various aspects of machine speech recognition, make it an ideal candidate for demonstrating the value of this data-transformation procedure for comparing multiple RPYS analyses concurrently.