Abstract
The Academic Journal Ranking Problem consists in formulating a formal assessment of scientific journals. An outcome variable must be constructed that allows valid journal comparison, either as a set of tiers (ordered classes) or as a numerical index. But part of the problem is also to devise a procedure to get this outcome, that is, how to get and use relevant data coming from expert opinions or from citations database. We propose a novel approach to the problem that applies fuzzy cluster analysis to peer reviews and opinion surveys. The procedure is composed of two steps: the first is to collect the most relevant qualitative assessments from international organizations (for example, the ones available in the Harzing database) and, as inductive analysis, to apply fuzzy clustering to determine homogeneous journal classes; the second deductive step is to determine the hidden logical rules that underlies the classification, using a classification tree to reproduce the same patterns of the first step.
Our approach is applied to the classification of 138 academic journals that were selected by members of AMASES, an Italian mathematics association, as the most prominent journals of our field. The clusters that are determined by our method show that rankings are affected by two hidden dimensions: one is the academic prestige of a publication, but the other is the disciplinary diffusion of a mathematics subfield. In particular, mathematics journals that are close to finance or economics are usually ranked better than journals dealing with linear algebra or systems dynamics.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
ADLER, M. and ZIGLIO, E. (Eds.) (1996), Gazing Into the Oracle: The Delphi Method and Its Application to Social Policy and Public Health, London: Kingsley Publishers.
ADLER, R., EWING, J., and TAYLOR, P. (2008), “Citation Statistics”, available at www.mathunion.org/fileadmin/IMU/CitationStatistics.pdf
BELACEL, N., HANSEN, P., and MLADENOVIC, N. (2002), “Fuzzy J-Means: A New Heuristic for Fuzzy Clustering”, Pattern Recognition, 35, 2193–2200.
BENATI, S., (2008), “Categorical Data Fuzzy Clustering: An Analysis of Local Search Heuristics”, Computers and Operations Research, 35, 766–775.
BEZDEK, J.C., KELLER, J., KRISNAPURAM, R., and PAL, N.R. (2005), Fuzzy Models and Algorithms for Pattern Recognition and Image Processing, New York: Springer-Verlag.
BREIMAN, L., FRIEDMAN, J., STONE, C.J., and OLSHEN, R.A. (1984), Classification and Regression Trees, City?: Chapman and Hall.
CAMERON, B.D. (2005), “Trends in the Usage of ISI Bibliometric Data: Uses, Abuses, and Implications”, Portal-Libraries and the Academy, 5, 105–125.
DONOHUE, J.A. and FOX, J.B. (2000), “A Multi-Method Evaluation of Journals in the Decision and Management Sciences by US Academics” Omega International Journal of Management Science, 28, 17–36.
DROBICS, M., BODENHOFER, U., and WINIWARTER, W. (2002), “Mining Clusters and Corresponding Interpretable Descriptions - A Three-Stage Approach”, Expert Systems, 19, 224–234.
FIGÀ TALAMANCA, A., (2000), “L’Impact Factors nella valutazione della ricerca”, available at http://siba2.unile.it/sinm/4sinm/interventi/fig-talam.htm
GONZALES, L. and CAMPANARIO, J.M. (2007), “Structure of the Impact Factor of Journals Included in the Social Sciences Citation Index: Citations from Documents Labeled ”Editorial Material”, Journal of the American Society for Information Science and Technology, 58, 252-262.
HARZING, A.W. (2005), “Journal Quality List, Version 2005”, available at http://www.harzing.com
HOPPNER, F., KRUSE, R., KLAWONN, F., and RUNKLER, T. (1999), Fuzzy Cluster Analysis: Methods for Classification, Data Analysis and Image Recognition, New York: John Wiley.
KALAITZIDAKIS, P., MAMUNEAS, T.P., and STENGOS, T. (2003), “Rankings of Academic Journals and Institutions in Economics” Journal of the European Economic Association, 16, 1346–1366.
LABAND, D. and PIETTE, M. (1994), “The Relative Impact of Economic Journals”, Journal of Economic Literature, 32, 640–666.
LUBRANO, M., BAUWENS, L., KIRMAN, A., and PROTOPOPESCU, C. (2003), “Ranking Economics Departments in Europe: A Statistical Approach” Journal of the European Economic Association, 16, 1367–1401.
MINGERS, J. and HARZING, A. (2006), “Ranking Journals in Business and Management: A Statistical Analysis in the Harzing Dataset”, available at www.harzing.com.
NG, M.K. and WONG, J.C. (2002), “Clustering Categorical Data Sets Using Tabu Search Techniques”, Pattern Recognition, 35, 2783–2790.
REEDIJK, J. (1998), “Sense and Nonsense of Science Citation Analyses: Comments on the Monopoly Position of ISI and Citation Inaccuracies. Risks of Possible Misuse and Biased Citation and Impact Data”, New Journal of Chemistry, 22, 767–770.
SCIMAGO WEB PAGE (2008), available at http://www.scimago.es
THOMSON SCIENTIFIC WEB PAGE, available at http://scientific.thomson.com/products/jcr/
TSE, A. (2001), “Using Mathematical Programming to Solve Large Ranking Problems”, Journal of the Operational Research Society, 52, 1144–1150.
ZHOU, D.N., MA H., and TURBAN, E. (2001), “Journal Quality Assessment: An Integrated Subjective and Objective Approach”, IEEE Transaction on Engineering Management, 48, 479–490.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Benati, S., Stefani, S. The Academic Journal Ranking Problem: A Fuzzy-Clustering Approach. J Classif 28, 7–20 (2011). https://doi.org/10.1007/s00357-011-9072-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00357-011-9072-1