The Hirsch spectrum: A novel tool for analyzing scientific journals
Introduction
At the present time, there is a wide number of scientific journals with different status, prestige and diffusion, covering innumerable scientific disciplines. The most well-known tool to evaluate scientific journals is the ISI impact factor (ISI-IF), which was introduced by Garfield (1972). This indicator allows comparisons among different journals, provided that they belong to the same subject area (Amin & Mabe, 2000). Although it shows some weak points, in many academic contexts it seems to be the main way for ranking journals (MacRoberts and MacRoberts, 1987, Seglen, 1992, Seglen, 1997, Jennings, 1998, Glänzel and Moed, 2002, Garfield, 2006, Brumback, 2008, Leydesdorff, 2009). Some main drawbacks of ISI-IF are: (i) not all scientific journals are indexed by Thomson Scientific, (ii) the limited time span (only citations accumulated within 2 years after the publication are considered) and (iii) the lack of coverage (citations in books, conference proceedings and dissertations are not included in the ISI list) (Seglen, 1997, Harzing, 2008, Thomson Reuters, 2009). Originally, ISI-IF was conceived to evaluate the diffusion of a journal but it had gradually become an indicator of prestige/reputation for the journal itself and, implicitly, for the authors of the papers there presented (Braun, Dióspatonyi, Zsindely, & Zádor, 2007). In practice, the larger ISI-IF, the more prestigious the journal.
For a potential author, the scientific reputation of the past and current authors of one journal is a reason of attraction. Reputation/prestige of the journal editor-in-chief and editorial board members, and presence of papers submitted by eminent scientists are some other possible reasons for preferring one journal to another. However, these evaluations are often subjective and not very reliable. Braun, Glänzel, & Schubert (2006) proposed using the Hirsch (h) index for evaluating and comparing scientific journals. Specifically, h is defined as the number such that, for a general group of papers, h papers received at least h citations while the other papers received no more than h citations (Hirsch, 2005, Hirsch, 2007). h was originally introduced by Hirsch in order to evaluate the quantity and the diffusion of one researcher's scientific production. Ever since its introduction, h received much attention. This indicator has many merits (i.e. it is synthetic, robust, simple to calculate and with immediate intuitive meaning) and some weak points; both have been abundantly pointed out in the literature (Moed, 2005, Egghe, 2006, Glänzel, 2006, Kelly and Jennions, 2006, Rousseau, 2006, Saad, 2006, Bornmann and Daniel, 2007, Costas and Bordons, 2007, Orbay et al., 2007, Schreiber, 2007, Van Raan, 2006, Wendl, 2007, Harzing and van der Wal, 2008, Mingers, 2009, Franceschini and Maisano, 2009a). Another tangible sign of the popularity of h is the appearance of many proposals for new variants and improvements, including the above-mentioned h-index for journals (Lehmann et al., 2005, Banks, 2006, Batista et al., 2006, Braun et al., 2006, Lehmann et al., 2006, BiHui et al., 2007, Burrell, 2007a, Burrell, 2007b, Castillo et al., 2007, Katsaros et al., 2007, Sidiropoulos et al., 2007, Schreiber, 2008, Antonakis and Lalive, 2008, Woeginger, 2008, Franceschini and Maisano, 2009b).
Coming back to the h-index for journals, it is calculated taking into consideration the articles published by a specific journal in a precise time period (e.g. 1 year). Unfortunately, this indicator has a significant limitation. Considering a generic journal, the citation accumulation process of the papers requires a certain amount of time to become stable—according to some authors, this period is about 5 years in the engineering field (Amin and Mabe, 2000, Castillo et al., 2007, Harzing, 2008). Thus, h for journals is not suitable to evaluate the most recently published journals and, much less, to compare them with other past journals. Besides, being sensitive to the number of papers per issue, this indicator – if calculated on a yearly basis – tends to favour journals with many papers/issues per year. In fact, a high number of articles per year are not necessarily an element in favour of a journal with respect to another.
The goal of this paper is to introduce the Hirsch spectrum (h-spectrum), a new tool that is derived from h and defined as the distribution representing the h-indexes associated to the authors of a specific journal, in a specific interval of time. The term “spectrum” is originated from the fact that this distribution provides an image of the author population of one journal for a period of interest. In our view, h-spectrum represents a different way for evaluating and comparing the reputation of journals (indexed by Thomson Scientific or not).
More in detail, h-spectrum can be used for several practical purposes, respectively:
- •
to make a comparison among journals within the same scientific field;
- •
to define the profile of the “typical authors” of a specific journal. This profile may represent a reference for other (potential) authors;
- •
extending the idea of the previous point, to define a reference of the “typical researcher” of a specific discipline (both in terms of productivity and diffusion, which are the basic reasons why we decided to use h);
- •
to help a journal's editorial board to periodically monitor the effect of the paper selection policy, from the point of view of the population of the journal authors. In this sense, h-spectrum may become an indirect indicator of editorial strategy;
- •
to provide a rough indication on the prestige/influence of a journal on the scientific community.
To focalize our preliminary analysis, the h-spectrum study is circumscribed to a particular discipline. We analyzed some journals in the Quality Engineering and Quality Management area.
The remaining of this paper is organised into three sections. Section 2 illustrates the methodology used in the analysis and shows some preliminary results. Section 3 focuses on some peculiar aspects of the h-spectrum and makes a brief comparison with ISI-IF. Section 4 identifies several ideas for further research activities, which may originate from this work. Finally, the conclusions are given, summarising the original contribution of this paper.
Section snippets
Methodology and preliminary results
The h-spectrum analysis can be divided in two distinct activities:
- 1.
construction and comparison of the h-spectra related to different journals in the same reference year, so as to investigate how the h-spectrum changes from journal to journal;
- 2.
construction and comparison of the h-spectra related to the same journal(s) in different periods of time, so as to investigate how a journal's h-spectrum tends to change over time.
Author's reputation
We think that h-spectrum can be a reliable tool for evaluating a journal at the very moment of the publication, despite the fact that it is based on the publications/citations accumulated before the publication of the examined journal. There are empirical proofs of the fact that citations that a new paper will receive in the future are generally consistent with the citations accumulated by previous papers of the same author, that is to say the author's reputation (Castillo et al., 2007). Being
Open issues
Several ideas for further research activities may originate from this work. Here follows a list of the most interesting ones:
- •
Repeating the analysis using other databases (i.e. Web of Science, Scopus or the DBLP digital library), so as to investigate possible differences in the results.
- •
Introducing a weighting system for author contribution, which takes account of multiple authorships and (co-)authors with multiple papers in the same period of interest, when determining the h-spectrum of a
Conclusions
The main novelty of this paper is the introduction and discussion of the h-spectrum, a new tool – based on the h-index – that can be used for three major purposes: (i) providing a reference for the (potential) authors of a scientific journal; (ii) performing rough comparisons between different journals within the same scientific field (journal academic reputation); (iii) helping a journal's editorial staff to periodically monitor the effect of the paper selecting policy.
The results of a
References (51)
On the h-index, the size of the Hirsch core and Jin's A-index
Journal of Informetrics
(2007)- et al.
The h-index: advantages, limitations and its relation with other bibliometric indicators at the micro level
Journal of Informetrics
(2007) - et al.
The h index and career assessment by numbers
Trends in Ecology & Evolution
(2006) An axiomatic characterization for the Hirsch-index
Mathematical Social Sciences
(2008)- American Society for Quality, (2009)...
- et al.
Impact Factors: Use and Abuse
(2000) - et al.
Quantifying scholarly impact: IQp versus the Hirsch h
Journal of the American Society for Information Science and Technology
(2008) An extension of the Hirsch index: indexing scientific topics and compounds
Scientometrics
(2006)- et al.
Is it possible to compare researchers with different scientific interests?
Scientometrics
(2006) - et al.
The R- and AR-indices: complementing the h-index
Chinese Science Bulletin
(2007)
What do we know about the h index?
Journal of the American Society for Information Science and Technology
A Hirsch-type index for journals
The Scientist
Gatekeeper index versus impact factor of science journals
Scientometrics
Worshipping false idols: the impact factor dilemma
Journal of Child Neurology
Hirsch index or Hirsch rate? Some thoughts arising from Liang's data
Scientometrics
Estimating number of citations using author reputation
String Processing and Information Retrieval
The H index distribution in the School of Public Health at the University of Sydney, November 2008
Theory and practise of the g-index
Scientometrics
Management by Measurement: Designing Key Indicators and Performance Measurement Systems
Properties of performance indicators in operations management: a reference framework
The International Journal of Productivity and Performance Management
Analysis of the Hirsch index's operational properties
European Journal of Operational Research
The Hirsch index in manufacturing and Quality engineering
Quality and Reliability Engineering International
Citation analysis as a tool in journal evaluation
Science
The history and meaning of the journal impact factor
JAMA
Journal impact measures in bibliometric research
Scientometrics
Cited by (28)
Topics and methods in economics, finance, and business journals: A content analysis enquiry
2018, HeliyonCitation Excerpt :This tool allows defining a reference profile of the typical author of a journal, comparison of different journals within the same scientific field, and a rough indication of prestige/reputation of a journal in the scientific community. h-Spectrum can be associated to every journal” (2010, pg. 66). In terms of citational analysis, many fields have listed the most influential journals and authors.
Growth dynamics of citations of cumulative papers of individual authors according to progressive nucleation mechanism: Concept of citation acceleration
2013, Information Processing and ManagementCitation Excerpt :In recent years, the h index proposed by Hirsch (2005) to quantify the research output of individual scientists has drawn constant attention in the academic literature. Apart from contributions dealing, among others, with improvement and modification of the h index (for example, see: Alonso, Cabrerizo, Herrera-Viedma, & Herrera, 2009; Anderson, Hankin, & Killworth, 2008; Burrell, 2009; Csajbók, Berhidi, Vasas, & Schubert, 2007; Egghe, 2010a, 2010b, 2010c; Franceschini & Maisano, 2010a; Franceschini & Maisano, 2010b; Glänzel & Schubert 2010; Jin, Liang, Rousseau, & Egghe, 2007; Kosmulski, 2006; Navon, 2009; Prathap, 2006; Schubert, 2007) and discussion on the relationship between different bibliometric evaluation measures (Burrell, 2009; Van Raan, 2006), attempts have been made to give mathematical models to the h index and its modifications and to investigate its dependence on time (Burrell, 2007a, 2007b, 2009; Egghe, 2007, 2008, 2009; Glänzel, 2008; Hirsch, 2005; Nair & Turlach, 2012; Ye & Rousseau, 2008). The main prediction of the deterministic model of Hirsch (2005) and the stochastic model of Burrell (2007a, 2007b) is that the ratio h(t)/t is expected to be a time-independent constant which, according to Hirsch (2005) “should provide a useful yardstick to compare scientists of different seniority”.
Structured evaluation of the scientific output of academic research groups by recent h-based indicators
2011, Journal of InformetricsCitation Excerpt :For this comparison to be as much exhaustive as possible, five different typologies of h-based indicators are used. They are respectively: the h-spectrum [Franceschini & Maisano, 2010b], the h-index of a research group (hGROUP) [Moed, 2005], the successive h-index of a research group (h2) [Schubert, 2007], the ch-index [Ajiferuke & Wolfram, 2010; Franceschini et al., 2010], and the h-index of single publications (hSINGLE) [Schubert, 2009]. We remark that analysis is focused on research groups, rather than whole universities.
The citation triad: An overview of a scientist's publication output based on Ferrers diagrams
2010, Journal of InformetricsCitation Excerpt :For example, these indicators can be distorted by self-citations or multi-authorship, they are not useful for cross-disciplinary comparisons, unfavourable to young scientists with few articles, etc. (Franceschini & Maisano, 2010b; Franceschini et al., in press).
Multilevel Graph-Based Decision Making in Big Scholarly Data: An Approach to Identify Expert Reviewer, Finding Quality Impact Factor, Ranking Journals and Researchers
2021, IEEE Transactions on Emerging Topics in Computing