Skip to main content
SpringerLink
Log in

Introducing sub-impact factor (SIF-) sequences and an aggregated SIF-indicator for journal ranking

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

The main methods for ranking academic journals are peer review based approaches and applications of various bibliometric indicators, or a mixture of the two. Such rankings are used to assess the overall quality of journals, although their real meaning remains unclear as long as the notion of “quality’ is not precisely defined. In our approach we examine journal evaluation from the perspective of knowledge accumulation taking the citation distribution into account. A new indicator, the sub-impact factor denoted as SIF, derived sub-impact factor sequences and an aggregated SIF-indicator are proposed. An empirical study is performed on 64 journals in the area of operations research and management science, illustrating the use of these indicators.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Aarssen, L. W., Tregenza, T., Budden, A. E., Lortie, C. J., Koricheva, J., & Leimu, R. (2008). Bang for your buck: Rejection rates and impact factors in ecological journals. The Open Ecology Journal, 1, 14–19.

    Article  Google Scholar 

  • Attaway, A. N., Baxendale, S. J., Foster, B. P., & Karcher, J. N. (2008). Reassessing accounting faculty scholarly expectations: Journal classification by author affiliation. Academy of Educational Leadership Journal, 12(3), 71–86.

    Google Scholar 

  • Bador, P., & Lafouge, T. (2010). Comparative analysis between impact factor and h-index for pharmacology and psychiatry journals. Scientometrics, 84(1), 65–79.

    Article  Google Scholar 

  • Bergstrom, C. (2007). Eigenfactor: Measuring the value and prestige of scholarly journals. C&RL News, 68(5), 314–316.

    Google Scholar 

  • Bontis, N., & Serenko, A. (2009). A follow-up ranking of academic journals. Journal of Knowledge Management, 13(1), 16–26.

    Article  Google Scholar 

  • Braun, T., Glänzel, W., & Schubert, A. (2006). A Hirsch-type index for journals. Scientometrics, 69(1), 169–173.

    Article  Google Scholar 

  • Brown, L. D. (2003). Ranking journals using Social Science Research Network downloads. Review of Quantitative Finance and Accounting, 20(3), 291–307.

    Article  Google Scholar 

  • Carter, C. R. (2002). Assessing logistics and transportation journals: Alternative perspectives. Transportation Journal, 42(2), 42–50.

    Google Scholar 

  • Centre National de la Recherche Scientifique Journal List. (2007). http://www.gredeg.cnrs.fr/Section37/Liste-2007-final.pdf.

  • Chandy, P. R., Ganesh, G. K., & Henderson, G. V. (1991). Awareness and evaluation of selected accounting journals inside and outside the discipline: An empirical study. Akron Business and Economic Review, 22(2), 214–226.

    Google Scholar 

  • Chen, C. R., & Huang, Y. (2007). Author affiliation index, finance journal ranking, and the pattern of authorship. Journal of Corporate Finance, 13(5), 1008–1026.

    Article  Google Scholar 

  • Christenson, J. A., & Sigelman, L. (1985). Accrediting knowledge: Journal stature and citation impact in social science. Social Science Quarterly, 66(4), 964–975.

    Google Scholar 

  • Cole, S., Cole, J. R., & Simon, G. A. (1981). Chance and consensus in peer review. Science, 214(4523), 881–886.

    Article  Google Scholar 

  • Cranfield UniversitySchool of Management Journal Rankings. (2006). http://www.som.cranfield.ac.uk/som/p10597/Research/Journal-Rankings.

  • Doyle, J. R., & Arthurs, A. J. (1995). Judging the quality of research in business schools: The UK as a case study. Omega, 23(3), 257–270.

    Article  Google Scholar 

  • Egghe, L. (2006). An improvement of the h-index: The g-index. ISSI Newsletter, 2(1), 8–9.

    MathSciNet  Google Scholar 

  • Erasmus Research Institute Journal List .(2006). http://www.erim.eur.nl/ERIM/About/EJL.

  • Extejt, M. M., & Smith, J. E. (1990). The behavioral sciences and management: An evaluation of relevant journals. Journal of Management, 16(3), 539–551.

    Article  Google Scholar 

  • Forgionne, G., & Kohli, R. (2001). A multiple criteria assessment of decision technology system journal quality. Information & Management, 38(7), 421–435.

    Article  Google Scholar 

  • Frandsen & Rousseau. (2005). Article impact calculated over arbitrary periods. Journal of the American Society for Information Science and Technology, 56(1), 58–62.

    Article  Google Scholar 

  • Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science, 178(4060), 471–479.

    Article  Google Scholar 

  • Garfield, E., & Sher, I. H. (1963). New factors in the evaluation of scientific literature through citation indexing. American Documentation, 14(3), 195–201.

    Article  Google Scholar 

  • Geary, J., Marriott, L., & Rowlinson, M. (2004). Journal rankings in business and management and the 2001 research assessment exercise in the UK. British Journal of Management, 15(2), 95–141.

    Article  Google Scholar 

  • Glänzel, W. (2011). The application of characteristic scores and scales to the evaluation and ranking of scientific journals. Journal of Information Science, 37(1), 40–48.

    Article  Google Scholar 

  • Glänzel, W., & Moed, H. F. (2002). Journal impact measures in bibliometric research. Scientometrics, 53(20), 171–193.

    Article  Google Scholar 

  • Harzing, A. W. (2007) Publish or Perish, available from http://www.harzing.com/pop.htm.

  • Haustein, S. (2012). Multidimensional journal evaluation. Berlin: De Gruyter Saur.

    Book  Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.

    Article  Google Scholar 

  • Holsapple, C. W. (2008). A publication power approach for identifying premier information systems journals. Journal of the American Society for Information Science and Technology, 59(2), 166–185.

    Article  Google Scholar 

  • Johnson, J. L., & Podsakoff, P. M. (1994). Journal influence in the field of management: An analysis using Salancik’s index in a dependency network. Academy of Management Journal, 37(5), 1392–1407.

    Article  Google Scholar 

  • Kochen, M. (1974). Principles of information retrieval. Los Angeles, CA: Melville Publishing.

    MATH  Google Scholar 

  • Krogstad, J., & Smith, G. (2003). Assessing the influence of auditing: A journal of practice and theory 1985–1999. Auditing, 22(1), 195–204.

    Article  Google Scholar 

  • Laband, D. N. (1990). Measuring the relative impact of economics book publishers and economics journals. Journal of Economic Literature, 28(2), 655–660.

    Google Scholar 

  • Leydesdorff, L., & Bornmann, L. (2011). Integrated impact indicators compared with impact factors: An alternative research design with policy implications. Journal of the American Society for Information Science and Technology, 62(11), 2133–2146.

    Article  Google Scholar 

  • Liang, L. M., & Rousseau, R. (2009). A general approach to citation analysis and an h-index based on the standard impact factor framework. In: ISSI 2009: 12th international conference on scientometrics and informetrics, (B. Larsen & J. Leta, eds.) (pp. 143–153), Rio de Janeiro: BIREME & Federal University of Rio de Janeiro.

  • Liebowitz, S. J., & Palmer, J. P. (1984). Assessing the relative impacts of economics journals. Journal of Economic Literature, 22(1), 77–88.

    Google Scholar 

  • Liyanage, C., Elhag, T., Ballal, T., & Li, Q. P. (2009). Knowledge communication and translation—A knowledge transfer model. Journal of Knowledge Management, 13(3), 118–131.

    Article  Google Scholar 

  • Ma, N., Guan, J., & Zhao, Y. (2008). Bringing PageRank to the citation analysis. Information Processing and Management, 44(2), 800–810.

    Article  Google Scholar 

  • Mingers, J., Macri, F., & Petrovici, D. (2012). Using the h-index to measure the quality of journals in the field of business and management. Information Processing and Management, 48(2), 234–241.

    Article  Google Scholar 

  • Moore, W. J. (1972). The relative quality of economics journals: A suggested rating system. Western Economic Journal, 10(2), 156–169.

    Google Scholar 

  • Morrison, A. J., & Inkpen, A. C. (1991). An analysis of significant contributions to the international business literature. Journal of International Business Studies, 22(1), 143–153.

    Article  Google Scholar 

  • Opthof, T. (1999). Submission, acceptance rate, rapid review system and impact factor. Cardiovascular Research, 41, 1–4.

    Article  Google Scholar 

  • Pinski, G., & Narin, F. (1976). Citation influence for journal aggregates of scientific publications: Theory, with application to the literature of physics. Information Processing and Management, 12(5), 297–312.

    Article  Google Scholar 

  • Polanyi, M. (1958). Personal knowledge: Towards a post-critical philosophy. London: Routledge & Kegan Paul.

    Google Scholar 

  • Prathap, G. (2010). The iCE approach for journal evaluation. Scientometrics, 85(2), 561–565.

    Article  Google Scholar 

  • Prather, J., & Rueschhoff, N. (1996). An analysis of international accounting research in U.S. Academic Accounting Journals, 1980 through 1993. Accounting Horizons, 10(1), 1–17.

    Google Scholar 

  • Rousseau, R. (1988). Citation distribution of pure mathematics journals. In L. Egghe & R. Rousseau (Eds.), Informetrics 87/88 (pp. 249–262). Amsterdam: Elsevier.

    Google Scholar 

  • Rousseau, R. (2002). Journal evaluation: Technical and practical issues. Library Trends, 50(3), 418–439.

    Google Scholar 

  • Rousseau, R., & Ye, F. Y. (2012). Basic independence axioms for the publication-citation system. Journal of Scientometric Research, 1(1), 22–27.

    Article  Google Scholar 

  • Rousseau, S. (2008). Journal evaluation by environmental and resource economists: A survey. Scientometrics, 77(2), 213–223.

    Article  Google Scholar 

  • Rousseau, S., Verbeke, T., & Rousseau, R. (2009). Evaluating environmental and resource economics journals: A TOP-curve approach. Review of Environmental Economics and Policy, 3(2), 270–287.

    Article  Google Scholar 

  • Saaty, T. L. (1980). The analytic hierarchy process. New York: McGraw-Hill.

    MATH  Google Scholar 

  • SCIMAGO. (2009). SCImago Institutions Rankings (SIR): 2009 World Report. http://www.scimagoir.com/.

  • Serenko, A., & Bontis, N. (2009). A citation based ranking of the business ethics scholarly journals. International Journal of Business Governance and Ethics, 4(4), 390–399.

    Article  Google Scholar 

  • Serenko, A., & Bontis, N. (2013). Global ranking of knowledge management and intellectual capital academic journals: 2013 update. Journal of Knowledge Management, 17(2), 307–326.

    Article  Google Scholar 

  • Serenko, A., & Jiao, C. Q. (2012). Investigating information systems research in Canada. Canadian Journal of administrative Sciences, 29(1), 3–24.

    Article  Google Scholar 

  • Skeels, J. W., & Taylor, R. A. (1972). The relative quality of economics journals: an alternative rating system. Western Economic Journal, 10(4), 470–473.

    Google Scholar 

  • Stahl, M. J., Leap, T. L., & Wei, Z. Z. (1988). Publication in leading management journals as a measure of institutional research productivity. Academy of Management Journal, 31(3), 707–720.

    Article  Google Scholar 

  • Tahai, A., & Meyer, M. (1999). A revealed preference study of management journal’s direct influences. Strategic Management Journal, 20(3), 279–296.

    Article  Google Scholar 

  • Testa, J. (2012). The Thomson Reuters journal selection process. http://wokinfo.com/essays/journal-selection-process/.

  • Vanclay, J. K. (2012). Impact factor: Outdated artefact or stepping-stone to journal certification? Scientometrics, 92(2), 211–238.

    Article  Google Scholar 

  • Vinkler, P. (2010). The evaluation of research by scientometric indicators. Oxford: Chandos Publishing.

    Book  Google Scholar 

  • Whitehouse, G. H. (2001). Citation rates and impact factors—Should they matter? British Journal of Radiology, 74(877), 1–3.

    Article  Google Scholar 

  • Xu, F. (2011). Measuring research output quality using bibliometric method within the HEFCE framework. Doctoral dissertation. University of Kent, Canterbury.

  • Xu, F., & Liu, W. B. (2013). Evaluation of SCI journals from the view of knowledge accumulation. Journal of the China Society for Scientific and Technical Information, 32(10), 1075–1089. (in Chinese).

    Google Scholar 

  • Yin, C.-Y., Aris, M. J., & Chen, X. (2010). Combination of Eigenfactor™ and h-index to evaluate scientific journals. Scientometrics, 84(3), 639–648.

    Article  Google Scholar 

Download references

Acknowledgments

Work by Fang Xu is supported by NSFC Grant No. 71201159, while work by Ronald Rousseau is supported by the NSFC Grants No. 71173154 and 71173185. The authors thank two reviewers for several useful suggestions.

Author information

Authors and Affiliations

  1. Institute of Policy and Management, Chinese Academy of Sciences, Beijing, 100190, China

    Fang Xu

  2. Kent Business School, University of Kent, Canterbury, CT2 7NZ, UK

    Wenbin Liu

  3. IOIW-IBW, University of Antwerp (UA), Venusstraat 35, 2000, Antwerp, Belgium

    Ronald Rousseau

  4. Department of Mathematics, KU Leuven, Celestijnenlaan, 3000, Louvain, Belgium

    Ronald Rousseau

Authors
  1. Fang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenbin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ronald Rousseau
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ronald Rousseau.

Appendix

Appendix

See Table 3.

Table 3 SIFew, SIFdw, JIF5, T (total number of citations), N (number of publications) and T/N (average number of citations) for 64 OR/MS journals over the period (2005–2009)
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, F., Liu, W. & Rousseau, R. Introducing sub-impact factor (SIF-) sequences and an aggregated SIF-indicator for journal ranking. Scientometrics 102, 1577–1593 (2015). https://doi.org/10.1007/s11192-014-1401-9

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-014-1401-9

Keywords

Search

Navigation