Elsevier

Journal of Informetrics

Volume 4, Issue 1, January 2010, Pages 64-73
Journal of Informetrics

The Hirsch spectrum: A novel tool for analyzing scientific journals

https://doi.org/10.1016/j.joi.2009.08.003Get rights and content

Abstract

This paper introduces the Hirsch spectrum (h-spectrum) for analyzing the academic reputation of a scientific journal. h-Spectrum is a novel tool based on the Hirsch (h) index. It is easy to construct: considering a specific journal in a specific interval of time, h-spectrum is defined as the distribution representing the h-indexes associated to the authors of the journal articles. This tool allows defining a reference profile of the typical author of a journal, compare different journals within the same scientific field, and provide a rough indication of prestige/reputation of a journal in the scientific community. h-Spectrum can be associated to every journal. Ten specific journals in the Quality Engineering/Quality Management field are analyzed so as to preliminarily investigate the h-spectrum characteristics.

Introduction

At the present time, there is a wide number of scientific journals with different status, prestige and diffusion, covering innumerable scientific disciplines. The most well-known tool to evaluate scientific journals is the ISI impact factor (ISI-IF), which was introduced by Garfield (1972). This indicator allows comparisons among different journals, provided that they belong to the same subject area (Amin & Mabe, 2000). Although it shows some weak points, in many academic contexts it seems to be the main way for ranking journals (MacRoberts and MacRoberts, 1987, Seglen, 1992, Seglen, 1997, Jennings, 1998, Glänzel and Moed, 2002, Garfield, 2006, Brumback, 2008, Leydesdorff, 2009). Some main drawbacks of ISI-IF are: (i) not all scientific journals are indexed by Thomson Scientific, (ii) the limited time span (only citations accumulated within 2 years after the publication are considered) and (iii) the lack of coverage (citations in books, conference proceedings and dissertations are not included in the ISI list) (Seglen, 1997, Harzing, 2008, Thomson Reuters, 2009). Originally, ISI-IF was conceived to evaluate the diffusion of a journal but it had gradually become an indicator of prestige/reputation for the journal itself and, implicitly, for the authors of the papers there presented (Braun, Dióspatonyi, Zsindely, & Zádor, 2007). In practice, the larger ISI-IF, the more prestigious the journal.

For a potential author, the scientific reputation of the past and current authors of one journal is a reason of attraction. Reputation/prestige of the journal editor-in-chief and editorial board members, and presence of papers submitted by eminent scientists are some other possible reasons for preferring one journal to another. However, these evaluations are often subjective and not very reliable. Braun, Glänzel, & Schubert (2006) proposed using the Hirsch (h) index for evaluating and comparing scientific journals. Specifically, h is defined as the number such that, for a general group of papers, h papers received at least h citations while the other papers received no more than h citations (Hirsch, 2005, Hirsch, 2007). h was originally introduced by Hirsch in order to evaluate the quantity and the diffusion of one researcher's scientific production. Ever since its introduction, h received much attention. This indicator has many merits (i.e. it is synthetic, robust, simple to calculate and with immediate intuitive meaning) and some weak points; both have been abundantly pointed out in the literature (Moed, 2005, Egghe, 2006, Glänzel, 2006, Kelly and Jennions, 2006, Rousseau, 2006, Saad, 2006, Bornmann and Daniel, 2007, Costas and Bordons, 2007, Orbay et al., 2007, Schreiber, 2007, Van Raan, 2006, Wendl, 2007, Harzing and van der Wal, 2008, Mingers, 2009, Franceschini and Maisano, 2009a). Another tangible sign of the popularity of h is the appearance of many proposals for new variants and improvements, including the above-mentioned h-index for journals (Lehmann et al., 2005, Banks, 2006, Batista et al., 2006, Braun et al., 2006, Lehmann et al., 2006, BiHui et al., 2007, Burrell, 2007a, Burrell, 2007b, Castillo et al., 2007, Katsaros et al., 2007, Sidiropoulos et al., 2007, Schreiber, 2008, Antonakis and Lalive, 2008, Woeginger, 2008, Franceschini and Maisano, 2009b).

Coming back to the h-index for journals, it is calculated taking into consideration the articles published by a specific journal in a precise time period (e.g. 1 year). Unfortunately, this indicator has a significant limitation. Considering a generic journal, the citation accumulation process of the papers requires a certain amount of time to become stable—according to some authors, this period is about 5 years in the engineering field (Amin and Mabe, 2000, Castillo et al., 2007, Harzing, 2008). Thus, h for journals is not suitable to evaluate the most recently published journals and, much less, to compare them with other past journals. Besides, being sensitive to the number of papers per issue, this indicator – if calculated on a yearly basis – tends to favour journals with many papers/issues per year. In fact, a high number of articles per year are not necessarily an element in favour of a journal with respect to another.

The goal of this paper is to introduce the Hirsch spectrum (h-spectrum), a new tool that is derived from h and defined as the distribution representing the h-indexes associated to the authors of a specific journal, in a specific interval of time. The term “spectrum” is originated from the fact that this distribution provides an image of the author population of one journal for a period of interest. In our view, h-spectrum represents a different way for evaluating and comparing the reputation of journals (indexed by Thomson Scientific or not).

More in detail, h-spectrum can be used for several practical purposes, respectively:

  • to make a comparison among journals within the same scientific field;

  • to define the profile of the “typical authors” of a specific journal. This profile may represent a reference for other (potential) authors;

  • extending the idea of the previous point, to define a reference of the “typical researcher” of a specific discipline (both in terms of productivity and diffusion, which are the basic reasons why we decided to use h);

  • to help a journal's editorial board to periodically monitor the effect of the paper selection policy, from the point of view of the population of the journal authors. In this sense, h-spectrum may become an indirect indicator of editorial strategy;

  • to provide a rough indication on the prestige/influence of a journal on the scientific community.

To focalize our preliminary analysis, the h-spectrum study is circumscribed to a particular discipline. We analyzed some journals in the Quality Engineering and Quality Management area.

The remaining of this paper is organised into three sections. Section 2 illustrates the methodology used in the analysis and shows some preliminary results. Section 3 focuses on some peculiar aspects of the h-spectrum and makes a brief comparison with ISI-IF. Section 4 identifies several ideas for further research activities, which may originate from this work. Finally, the conclusions are given, summarising the original contribution of this paper.

Section snippets

Methodology and preliminary results

The h-spectrum analysis can be divided in two distinct activities:

  • 1.

    construction and comparison of the h-spectra related to different journals in the same reference year, so as to investigate how the h-spectrum changes from journal to journal;

  • 2.

    construction and comparison of the h-spectra related to the same journal(s) in different periods of time, so as to investigate how a journal's h-spectrum tends to change over time.

Author's reputation

We think that h-spectrum can be a reliable tool for evaluating a journal at the very moment of the publication, despite the fact that it is based on the publications/citations accumulated before the publication of the examined journal. There are empirical proofs of the fact that citations that a new paper will receive in the future are generally consistent with the citations accumulated by previous papers of the same author, that is to say the author's reputation (Castillo et al., 2007). Being

Open issues

Several ideas for further research activities may originate from this work. Here follows a list of the most interesting ones:

  • Repeating the analysis using other databases (i.e. Web of Science, Scopus or the DBLP digital library), so as to investigate possible differences in the results.

  • Introducing a weighting system for author contribution, which takes account of multiple authorships and (co-)authors with multiple papers in the same period of interest, when determining the h-spectrum of a

Conclusions

The main novelty of this paper is the introduction and discussion of the h-spectrum, a new tool – based on the h-index – that can be used for three major purposes: (i) providing a reference for the (potential) authors of a scientific journal; (ii) performing rough comparisons between different journals within the same scientific field (journal academic reputation); (iii) helping a journal's editorial staff to periodically monitor the effect of the paper selecting policy.

The results of a

References (51)

  • L. Bornmann et al.

    What do we know about the h index?

    Journal of the American Society for Information Science and Technology

    (2007)
  • T. Braun et al.

    A Hirsch-type index for journals

    The Scientist

    (2006)
  • T. Braun et al.

    Gatekeeper index versus impact factor of science journals

    Scientometrics

    (2007)
  • R. Brumback

    Worshipping false idols: the impact factor dilemma

    Journal of Child Neurology

    (2008)
  • Q.L. Burrell

    Hirsch index or Hirsch rate? Some thoughts arising from Liang's data

    Scientometrics

    (2007)
  • C. Castillo et al.

    Estimating number of citations using author reputation

    String Processing and Information Retrieval

    (2007)
  • S. Chapman

    The H index distribution in the School of Public Health at the University of Sydney, November 2008

    (2009)
  • L. Egghe

    Theory and practise of the g-index

    Scientometrics

    (2006)
  • F. Franceschini et al.

    Management by Measurement: Designing Key Indicators and Performance Measurement Systems

    (2007)
  • F. Franceschini et al.

    Properties of performance indicators in operations management: a reference framework

    The International Journal of Productivity and Performance Management

    (2008)
  • F. Franceschini et al.

    Analysis of the Hirsch index's operational properties

    European Journal of Operational Research

    (2009)
  • F. Franceschini et al.

    The Hirsch index in manufacturing and Quality engineering

    Quality and Reliability Engineering International

    (2009)
  • E. Garfield

    Citation analysis as a tool in journal evaluation

    Science

    (1972)
  • E. Garfield

    The history and meaning of the journal impact factor

    JAMA

    (2006)
  • W. Glänzel et al.

    Journal impact measures in bibliometric research

    Scientometrics

    (2002)
  • Cited by (28)

    • Topics and methods in economics, finance, and business journals: A content analysis enquiry

      2018, Heliyon
      Citation Excerpt :

      This tool allows defining a reference profile of the typical author of a journal, comparison of different journals within the same scientific field, and a rough indication of prestige/reputation of a journal in the scientific community. h-Spectrum can be associated to every journal” (2010, pg. 66). In terms of citational analysis, many fields have listed the most influential journals and authors.

    • Growth dynamics of citations of cumulative papers of individual authors according to progressive nucleation mechanism: Concept of citation acceleration

      2013, Information Processing and Management
      Citation Excerpt :

      In recent years, the h index proposed by Hirsch (2005) to quantify the research output of individual scientists has drawn constant attention in the academic literature. Apart from contributions dealing, among others, with improvement and modification of the h index (for example, see: Alonso, Cabrerizo, Herrera-Viedma, & Herrera, 2009; Anderson, Hankin, & Killworth, 2008; Burrell, 2009; Csajbók, Berhidi, Vasas, & Schubert, 2007; Egghe, 2010a, 2010b, 2010c; Franceschini & Maisano, 2010a; Franceschini & Maisano, 2010b; Glänzel & Schubert 2010; Jin, Liang, Rousseau, & Egghe, 2007; Kosmulski, 2006; Navon, 2009; Prathap, 2006; Schubert, 2007) and discussion on the relationship between different bibliometric evaluation measures (Burrell, 2009; Van Raan, 2006), attempts have been made to give mathematical models to the h index and its modifications and to investigate its dependence on time (Burrell, 2007a, 2007b, 2009; Egghe, 2007, 2008, 2009; Glänzel, 2008; Hirsch, 2005; Nair & Turlach, 2012; Ye & Rousseau, 2008). The main prediction of the deterministic model of Hirsch (2005) and the stochastic model of Burrell (2007a, 2007b) is that the ratio h(t)/t is expected to be a time-independent constant which, according to Hirsch (2005) “should provide a useful yardstick to compare scientists of different seniority”.

    • Structured evaluation of the scientific output of academic research groups by recent h-based indicators

      2011, Journal of Informetrics
      Citation Excerpt :

      For this comparison to be as much exhaustive as possible, five different typologies of h-based indicators are used. They are respectively: the h-spectrum [Franceschini & Maisano, 2010b], the h-index of a research group (hGROUP) [Moed, 2005], the successive h-index of a research group (h2) [Schubert, 2007], the ch-index [Ajiferuke & Wolfram, 2010; Franceschini et al., 2010], and the h-index of single publications (hSINGLE) [Schubert, 2009]. We remark that analysis is focused on research groups, rather than whole universities.

    • The citation triad: An overview of a scientist's publication output based on Ferrers diagrams

      2010, Journal of Informetrics
      Citation Excerpt :

      For example, these indicators can be distorted by self-citations or multi-authorship, they are not useful for cross-disciplinary comparisons, unfavourable to young scientists with few articles, etc. (Franceschini & Maisano, 2010b; Franceschini et al., in press).

    View all citing articles on Scopus
    View full text