Skip to main content
SpringerLink
Log in

Relative measure index: a metric to measure the quality of journals

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Journal impact factors (JIF) have been an accepted indicator of ranking journals. However, there has been increasing arguments against the fairness of using the JIF as the sole ranking criteria. This resulted in the creation of many other quality metric indices such as the h-index, g-index, immediacy index, Citation Half-Life, as well as SCIMago journal rank (SJR) to name a few. All these metrics have their merits, but none include any great degree of normalization in their computations. Every citation and every publication is taken as having the same importance and therefore weight. The wealth of available data results in multiple different rankings and indexes existing. This paper proposes the use of statistical standard scores or z-scores. The calculation of the z-scores can be performed to normalize the impact factors given to different journals, the average of z-scores can be used across various criteria to create a unified relative measurement (RM) index score. We use the 2008 JCR provided by Thompson Reuters to demonstrate the differences in rankings that would be affected if the RM-index was adopted discuss the fairness that this index would provide to the journal quality ranking.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abbasi, A., Altmann, J., & Hwang, J. (2010). Evaluating scholars based on their academic collaboration activities: two indices, the RC-index and the CC-index, for quantifying collaboration activities of researchers and scientific communities. Scientometrics, 83(1), 1–13.

    Article  Google Scholar 

  • Adler, R., Ewing, J. & Taylor, P. (2008). Citation Statistics. Retrieved February 24, 2009, from http://www.ams.org/ewing/Documents/CitationStatistics-FINAL-1.pdf.

  • Amin, M., & Mabe, M. (2000). Impact factor: use and abuse. Perspectives in Publishing, 1, 1–6.

    Google Scholar 

  • Archambault, E., & Gagne, E. V. (2004). The use of bibliometrics in social sciences and humanities. Montreal: Social Sciences and Humanities Research Council of Canada (SSHRCC).

    Google Scholar 

  • Bornmann, L., Mutz, R., Neuhaus, C., & Daniel, H. D. (2008). Citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics in Science and Environmental Politics, 8(1), 93–102.

    Article  Google Scholar 

  • Campanario, J. M., Carretero, J., Marangon, V., Molina, A., & Ros, G. (2011). Effect on the journal impact factor of the number and document type of citing records: a wide-scale study. Scientometrics, 87, 75–84.

    Article  Google Scholar 

  • Campbell, P. (2008). Escape from the impact factor. Ethics in Science and Environmental Politics, 8(1), 5–6.

    Article  Google Scholar 

  • Egghe, L. (2006). Theory and practice of the g-index. Scientometrics, 69, 131–152.

    Article  Google Scholar 

  • Fok, D., & Franses, P. H. (2007). Modeling the diffusion of scientific publications. Journal of Econometrics, 139(2), 376–390.

    Article  MathSciNet  Google Scholar 

  • Franceshet, M. (2010). Journal influence factors. Journal of Informatics, 4(3), 239–248.

    Google Scholar 

  • Frandsen, T., Rousseau, R., & Rowlands, I. (2006). Diffusion factors. Journal of Documentation, 62(1), 58–72.

    Article  Google Scholar 

  • Garfield, E. (1999). Journal impact factor: a brief review. Canadian Medical Association Journal, 161(8), 979–980. http://www.cmaj.ca/cgi/content/full/161/8/979?ijkey=nr8.IXo1aXxvc 19 October.

    Google Scholar 

  • Garfield, E. (2005). The agony and the ecstasy: the history and meaning of the journal impact factor. International Congress on Peer Review and Biomedical Publication. http://garfield.library.upenn.edu/papers/jifchicago2005.pdf. Accessed 1 May 2009.

  • Glanzel, W. (2006). On the h-index—a mathematical approach to a new measure of publication activity and citation impact. Scientometrics, 67(2), 315–321.

    Article  Google Scholar 

  • Gonzalez-Pereira, B., Guerrero-Bote, V. P., & Moya-Anegon, F. (2010). A new approach to the metric of journal scientific prestige: the SJR indicator. Journal of Informetrics, 4(3), 379–391. Accessed at http://www.sciencedirect.com/science/article/pii/S1751157710000246

    Article  Google Scholar 

  • Guns, R., Liu, Y. X., & Mahbuba, D. (2011). Q-measures and betweenness centrality in a collaboration network: a case study of the field of informetrics. Scientometrics, 87(1), 133–147.

    Article  Google Scholar 

  • Haddow, G., & Genoni, P. (2009). Australian education journals: quantitative and qualitative indicators. Australian Academic & Research Libraries, 40(2), 88–104.

    Google Scholar 

  • Harnad, S. (2008). Validating research performance metrics against peer rankings. Ethics in Science and Environmental Politics, 8(1), 103–107.

    Article  MathSciNet  Google Scholar 

  • Harnad, S. & Brody, T. (2004). Comparing the impact of open access (OA) vs non-OA articles in the same journals. D-Lib Magazine, 10(6), (June 2004). Retrieved from: http://www.dlib.org/dlib/june04/harnad/06harna.html. Accessed 13 Oct 2011.

  • Harter, S. P. (1998). Scholarly communications and electronic journals: an impact Study. Journal of the American Society of Information Science, 49(6), 507–516.

    Article  Google Scholar 

  • Harzing, A.W. & Van der Wal, R. (2007). Google scholar: the democratization of citation analysis? Ethics in Science and Environmental Politics. Retrieved from: http://imechanica.org/files/gsdemo.pdf. Accessed 13 Oct 2011.

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences, USA, 102(46), 16569–16572.

    Article  Google Scholar 

  • Hodge, D. R., & Lacasse, J. R. (2011). Evaluating journal quality: is the H index a better measure than impact factors. Research on Social work Practice, 21(2), 222–230.

    Article  Google Scholar 

  • Korn, A., Schubert, A., & Telcs, A. (2009). Lobby index in networks. Physica A: Statistical Mechanics and its Applications, 388, 2221–2226.

    Article  Google Scholar 

  • Lee, K. P., Schotland, M., Bacchetti, Peter., & Bero, L. A. (2002). Association of journal quality indicators with methodological quality of clinical research articles. Journal of American Medical Association, 287(21), 2805–2808.

    Article  Google Scholar 

  • Monastersky, R. (2005). The number that’s devouring science. Chronicle of Higher Education, 52(8), A12–A17.

    Google Scholar 

  • Pringle, J. (2004). Do open access journals have impact. Nature. Retrieved from: http://www.nature.com/nature/focus/access/debate/19.html. Accessed 13 Oct 2011.

  • Rowlands, I. (2002). Journal diffusion factors: a new approach to measuring research influence. Aslib Proceedings, 54(2), 77–84.

    Article  Google Scholar 

  • Saha, S., Saint, C., & Christakis, D. R. (2003). Impact factor: a valid measure of journal quality? Journal of Medical Librarian Association, 91(1), 42–46.

    Google Scholar 

  • Schubert, A., Korn, A., & Telcs, A. (2009). Hirsch-type indices for characterizing networks. Scientometrics, 78(2), 375–382.

    Article  Google Scholar 

  • Sharma, O. (2007). Journal impact factor: an essential quality indicator. Current Science, 23(3), 141–142.

    Google Scholar 

  • Thomson Scientific (2009). 2008 Journal Citation Report. Available: http://wokinfo.com/products_tools/analytical/jcr. Accessed on: 1 Dec 2011.

  • Thomson Scientific (2011a). Thompson Reuters Journal Selection Process. Available: http://thomsonreuters.com/products_services/science/free/essays/journal_selection_process. Accessed on 2 Feb 2012.

  • Thomson Scientific (2011b). The Thompson Reuters Impact Factor. Available: http://thomsonreuters.com/products_services/science/free/essays/impact_factor. Accessed on: 2 Feb 2012.

  • Todd, P. A., & Ladle, R. J. (2008). Hidden dangers of a ‘citation culture’. Ethics in Science and Environmental Politics, 8(1), 13–16.

    Article  Google Scholar 

  • Zhao, S. X., Rousseau, R., & Ye, F. Y. (2011). h-Degree as a basic measure in weighted networks. Journal of Informetrics, 5, 668–677.

    Article  Google Scholar 

Download references

Acknowledgment

This study was made possible from consultation funding provided by the Malaysian Citation Centre, Ministry of Higher Education Malaysia addressed at the Faculty of Computer Science and Information Technology, University of Malaya, Kuala Lumpur, Malaysia.

Author information

Authors and Affiliations

  1. Department of Artificial Intelligence, Faculty of Computer Science and Information Technology, University of Malaya, Kuala Lumpur, Malaysia

    R. G. Raj

  2. Department of Library & Information Science, Malaysian Citation Centre, Faculty of Computer Science and Information Technology, University of Malaya, Kuala Lumpur, Malaysia

    A. N. Zainab

Authors
  1. R. G. Raj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. N. Zainab
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to R. G. Raj.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Raj, R.G., Zainab, A.N. Relative measure index: a metric to measure the quality of journals. Scientometrics 93, 305–317 (2012). https://doi.org/10.1007/s11192-012-0675-z

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-012-0675-z

Keywords

Search

Navigation