Journal acceptance rates: A cross-disciplinary analysis of variability and relationships with journal measures

Highlights

• Acceptance rates vary statistically significantly by discipline of journal, country affiliation of the editor, and number of reviewers per article.

• There is a negative relationship between acceptance rates and citation-based indicators.

• There is a positive relationship between acceptance rates and journal age.

• Open access journals have significantly higher mean acceptance rates than non-open access journals.

Abstract

There are many indicators of journal quality and prestige. Although acceptance rates are discussed anecdotally, there has been little systematic exploration of the relationship between acceptance rates and other journal measures. This study examines the variability of acceptance rates for a set of 5094 journals in five disciplines and the relationship between acceptance rates and JCR measures for 1301 journals. The results show statistically significant differences in acceptance rates by discipline, country affiliation of the editor, and number of reviewers per article. Negative correlations are found between acceptance rates and citation-based indicators. Positive correlations are found with journal age. These relationships are most pronounced in the most selective journals and vary by discipline. Open access journals were found to have statistically significantly higher acceptance rates than non-open access journals. Implications in light of changes in the scholarly communication system are discussed.

Introduction

The scholarly publication system operates on the basis of exchange. As in any market, there are suppliers (authors) and buyers² (journals) of goods (papers). In this exchange, authors are seeking journals with the highest impact in order to increase their stock of symbolic capital (Bourdieu, 1984), while journals attempt to capture papers that will increase their prestige. As a general rule, the best authors want to publish in the best journals, and the best journals want the best authors (those with the potentially best papers) to publish with them. In a perfect (i.e., optimally efficient) market, the best papers would gravitate to the best journals (Oster, 1980). But in this as so many other markets both suppliers and buyers lack perfect information. Absent perfect information, the various actors involved rely upon a range of indicators (bibliometric, sociometric, and demographic) and tacit knowledge to guide decision-making. One such indicator is acceptance rate, the proportion of papers submitted to a journal that are subsequently accepted and published.

Space at the upper end of the market is highly sought after and in limited supply. Competition and ambition often drive scholars to submit papers to journals beyond their reach, creating a cascade of rejected papers that puts added pressure on reviewers and editors (Craig, 2010, Cronin and McKenzie, 1992, Kravitz and Baker, 2011). Economic models have been proposed to analyze, inter alia, research spillover effects, duality in scientific discovery and congestion in information processing (Besancenot, Huynh, & Vranceanu, 2009, p. 1). Such models highlight the “informational frictions” that occur when papers are being matched with journals (Besancenot et al., 2009, p. 2).

Peer review is the established mechanism for allocating space to papers within a journal. Experts (editors, editorial board members and external reviewers) assess the quality of submitted papers and evaluate their suitability for publication. It is assumed that editors and reviewers are unbiased in their assessments and that the governing norm of impartiality is not violated, at least not egregiously (Lee et al., 2013, Sugimoto and Cronin, 2013). In reality, it is not quite so straightforward, as variations in consensus as to what constitutes quality, broadly conceived, within and across fields, can have an effect on acceptance rates (Hargens, 1988, Kravitz et al., 2010).

Variation in journal acceptance rates is an understudied area, not the least because of the difficulty in obtaining reliable data. One of the most comprehensive (and earliest) studies to date examined the rejection rates of 83 journals across a broad spectrum of disciplinary areas and found that humanities and social science journals have the highest and the biological sciences the lowest rates of rejection (Zuckerman & Merton, 1971, p. 77): “the more humanistically oriented the journal, the higher the rate of rejecting manuscripts for publication; the more experimentally oriented, with an emphasis on rigor of observation and analysis, the lower the rate of rejection.” Subsequent monodisciplinary studies have confirmed these findings (e.g., Cherkashin et al., 2009, Rotton et al., 1993, Schultz, 2010, Seaton, 1975, Vlachy, 1981). One explanation for this is the degree to which a dominant paradigm exists in any given discipline, providing a consensus as to what constitutes valid research (Kuhn, 1970).

It has been noted that there exists little guidance for calculating acceptance rates (Moore & Perry, 2012). At face value, the calculation may seem simple enough—the number of papers accepted over the total number of papers submitted. However, this is complicated by the unreliability of self-report data, inconsistent definitions of a resubmission, the inclusion/exclusion of invited papers or special issues in the calculations, the timeframe used, and the inclusion/exclusion of book reviews, among other considerations (Moore & Perry, 2012). Additionally, many studies rely on individual surveys of editors/publishers, rather than using a standard source for evaluation. Cabell's Directories of Publishing Opportunities (Cabell's henceforth) is one such source, but has been used only rarely in empirical studies (e.g., Haensly, Hodges, & Davenport, 2008).

Acceptance rates ostensibly testify to the relative competitiveness of a journal and have been used as a quality indicator. Statistically significant negative correlations between acceptance rates and other proxies of quality (i.e., citation rates, Journal Impact Factor [JIF]) have been demonstrated (Buffardi and Nichols, 1981, Haensly et al., 2008, Lee et al., 2002). Rotton et al. (1993) found that rejection rates were good predictors of citations, while Haensly et al. (2008) found acceptance rates to be statistically significantly correlated with both citations and survey-based rankings of journals. However, and with few exceptions, these have relied on small scale and monodisciplinary datasets and are somewhat dated.

More comprehensive studies are necessary to elucidate the relationship between acceptance rates and other indicators. The results of such studies can be used to assess the utility of journal acceptance rates and the degree to which these can be considered appropriate proxies for quality. To this end, the present paper provides the largest study of acceptance rates to date and investigates the following research questions and associated hypotheses:

• What is the degree of variability in acceptance rates?

  H1. Statistically significant differences in acceptance rates will be observable by discipline.

  H2. Statistically significant differences in acceptance rates will be observable by country of editor's location.

  H3. Statistically significant differences in acceptance rates will be observable by number of external reviewers.

  H4. Statistically significant differences in acceptances rates will be observable between JCR and non-JCR journals.

• What is the relationship between acceptance rates and JCR measures?

  H5. Statistically significant negative correlations will be observable between acceptance rates and citation-based JCR measures.

  H6. Statistically significant positive correlations will be observable between acceptance rates and size-based JCR measures (e.g., number of publications).

• What is the relationship between acceptance rates and age of journal?

  H7. Statistically significant negative relationships will be observable between acceptance rates and age of journal.

• What is the relationship between acceptance rates and the open access status of journals?

  H8. Open access journals will have statistically significantly higher acceptance rates.

The results of this analysis will inform scientometricians, policy makers, and scholars who employ these metrics to make decisions about where to publish and the relative quality of venues.