Review of Beyond Bibliometrics: Harnessing Multidimensional Indicators of Scholarly Impact

Bibliometrics is a fast evolving research field and at the same time a widespread but contested method for evaluating research. The recently published volume Beyond Bibliometrics: Harnessing Multidimensional Indictors of Scholarly Impact (edited by Blaise Cronin and Cassidy Sugimoto) provides a multifaceted picture of the current state of bibliometric research. In over 400 pages and 21 chapters it covers an array of issues stretching from the history of the field, ethical issues, development of new (alt)metric methods to descriptions of advanced methodologies for mapping and evaluating research. The volume certainly captures the heterogeneity of bibliometric research, and a multifold of perspectives and methods are presented.

The book begins with an inspiring overview by Blaise Cronin in which he paints an expansive picture of the field in a time when “[i]nterest in bibliometrics, broadly constructed, has never been greater” (p. 5). The introduction is followed by a historical account of the field written by Nicola de Bellis. His chapter shows how the ideas underpinning bibliometric research can be traced back to the scientific revolution and the rise of mathematical thinking. Thereafter he follows the development of the field from the English eugenicist Francis Galton and French documentalist Paul Otlet on to more familiar names as Robert K. Merton and Eugene Garfield. This investigation into the deep, and sometimes uncomforting roots of the field provides an important backdrop for the rest of the volume.

The section “critiques” is a much welcome and valuable addition to the bibliometric literature, especially as the issues raised here are seldom discussed in the journal literature. The call for critique and ethical standards suggests that bibliometricians have now realized that bibliometric measures play an increasingly important role in academia. Paul Wouters thought-provoking contribution, The Citation: From Culture to Infrastructure, does in fact suggest that citations are now part of “[…] the taken-for-granted context that enables our life and work” (p. 61). The citation has become an institutionalized part of academic life, although its status as an indicator of “quality” is highly contested. Ronald Day, on the other hand, focusses on the effects that further measurements and data have on our identity as scholars. Are we (as researchers) more than the data that describes us? This question becomes central when researchers have direct access to services such as Google Scholar, which in an instance calculates the number of citations that your publications have received. Yves Gingras and Jonathan Furner both recognize the influence that bibliometric measurements and rankings has on academia and in their contributions they develop guidelines and ethical standards for bibliometric assessment. By taking two familiar examples – the Shanghai-ranking and the h-index – Gingras shows in a straightforward manner what happens when indicators do not measure what they intend to measure. For example, a high h-index might in some cases indicate that your publications have had a certain impact, but as it is dependent on the number of published items it is inherently biased toward older researchers. Furner’s ethical considerations focus less on the actual methods used but rather on the intensions and values that are associated with bibliometric measurement. The ethical perspective is indeed valuable, and the actual need for a code of ethics highlights tensions between basic research and commercial interests that have always characterized the field. Together these chapters provide insightful critiques of current bibliometric practice and the concerns raised here are hopefully only the beginning of a more critical and self-reflective strand of research.

Bibliometric research is often geared toward the construction and refinement of methods and tools for studying the development of science. It is therefore not surprising that several chapters focus on new techniques and approaches, and the volume also covers state-of-the-art methodology in the field. The book offers thorough introductions into core topics such as science visualization, bibliometric evaluation, identification of research strengths and recommendation systems. A noteworthy contribution in the section on tools and methods is Vincent Larivière and Yves Gingras’ study of interdisciplinarity, which convincingly shows that although interdisciplinarity has risen since the 1980s, it is still not much higher today than it was in the 1930s. Furthermore, interdisciplinarity appears to be related to the availability of resources, as the level of interdisciplinarity rises when money for research is scarce. Cassidy Sugimoto’s chapter on academic genealogy is also worth mentioning, and her focus on supervisors and their disciples appears as a promising venue of research for a field that increasingly opens up for analyzes that go beyond traditional academic publications.

The effort to broaden the field is perhaps most visible in the new rapidly emerging subfield of “altmetrics.” Alternative metrics is one of the main themes of the volume, which is also reflected in the title “Beyond bibliometrics.” This theme starts out with an introductory piece by the man who coined the term “altmetrics,” Jason Priem. He defines altmetrics as “[…] the study and use of scholarly impact measures based on activity in online tools and environments” (p. 266). Services that are commonly used in altmetric research include Twitter, reference managers (e.g. Mendeley) as well as social networking sites such as Facebook. Priem argues that altmetrics have many advantages compared to traditional metrics – speed, breadth and diversity – while lack of theory, ease of gaming and bias toward young researchers with a high web presence seriously limit their use. The section on altmetrics continues to cover a whole range of new approaches including readership metrics as well as web impact metrics and the reader is given a comprehensive introduction to these emerging methods. Yet, several contributions in this section overlap and a more balanced stance toward altmetric measures would have been welcome. For an example one could ask how altmetrics fit into descriptions of a general evaluation culture in academia. And how do we deter them from becoming “tools of narcissism”? (Wouters and Costas, 2012). The optimism surrounding these new measures is not unwarranted but a more critical view, perhaps inspired by the perspective provided by Day, could have taken the discussion one step further.

The volume, which incorporates contributions from a range of leading scholars in bibliometric research, also paints a portrait of the research field as such. In his contribution Ronald Day writes: “From a little recognized field, 20th-century documentation, and following this more recently, a very small, largely Anglo-American, discipline, ‘library and information science,’ an entire episteme, indeed, an entire metaphysics of identity and power has arisen” (p. 80). Even if one doesn’t fully agree with this description it is tempting to reflect on the status and position of the research field of bibliometrics. Information scientists usually perceive bibliometrics as a subfield to Library and Information Science (LIS), and research on the topic is published in information science journals. However, few contributions – except the historical accounts by Blaise Cronin and Nicola De Bellis – touch on the connection between bibliometrics and LIS, and when scrutinizing the bios of contributing authors one can easily determine that the field of bibliometrics is indeed a multidisciplinary one. A recent study also show that one of the major LIS journals, JASIS, gradually is turning into a specialty journal for bibliometrics rather than a periodical for general research in LIS (see Nicolaisen and Frandsen, 2014). Thus, it could well be that the former subfield of bibliometrics is now outgrowing its “parent field” and the consequences that this might have for education and research in LIS are yet to be revealed.

Beyond bibliometrics provides a comprehensive and multifaceted overview of the field of bibliometrics. Yet, two themes stand out: the further development of altmetric methods and the emphasis of critical perspectives on the use of bibliometric measures. This signals two emergent trends in the field; first, a further extension and broadening of perspectives, methods and materials, and second, an increasing reflexiveness and awareness regarding the considerable influence that bibliometric measures have on research and higher education. In fact, these lines of inquiry should accompany one another in order for the field of bibliometrics to grow and develop while maintaining its integrity and significance as a field of research.

Beyond bibliometrics stands out through the breadth of its topics, yet I find that some perspectives are missing and the most obvious one is the aspect of gender. The term gender is not even in the index, and the omission of this perspective is surprising, as research on academic careers and gender differences is a growing and highly relevant topic for bibliometric research. Little is also said about research on books and patents. However, it would be impossible to cover the entire field and the focus on emerging topics has its merits. In all, the volume offers a multifold of insightful contributions by leading researchers, and the often clear and enjoyable way in which many contributions is written makes the volume stand out. I highly recommend it to everyone with the slightest interest in bibliometrics and scholarly communication.