Abstract
Numerous attempts have been made to validate the use of citation as an evaluation method by comparing it with peer review. Unlike past studies using journals, research articles or universities as the subject matter, the present study extends the comparison to the ranking of individual scholars. Results show that citation ranking and expert judgment of communication scholars are highly correlated. The citation method and the expert judgment method are found to work better in smaller research areas and yield more valid evaluation results for more prominent scholars.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes and references
R. C. Anderson, F. Narin, P. McAllister, Publication ratings versus peer ratings of universities,Journal of the American Society for Information Science, 29 (1978) 91–103.
M. E. D. Koenig, Determinants of expert judgement of research performance,Scientometrics, 4 (1982) 361–378.
M. E. D. Koenig, Bibliometric indicators versus expert opinion in assessing research performance,Journal of the American Society for Information Science, 34 (1983) 136–145.
S. M. Lawani, A. E. Bayer, Validity of citation criteria for assessing the influence of scientific publications: New evidence with peer assessment,Journal of the American Society for Information Science, 34 (1983) 59–66.
A. L. Porter, D. E. Chubin, X. Jin, Citations and scientific progress: Comparing bibliometric measures with scientific judgments,Scientometrics, 13 (1988) 103–124.
P. R. McAllister, R. C. Anderson, F. Narin, Comparison of peer and citation assessment of the influence of scientific journals,Journal of the American Society for Information Science, 31 (1980) 147–152.
M. D. Gordon, Citation ranking versus subjective evaluation in the determination of journal hierarchies in the social sciences,Journal of the American Society for Information Science, 33 (1982) 55–57.
A. Singleton, Journal ranking and selection: A review in physics,Journal of Documentation, 32 (1976) 258–289.
H. H. Garrison, S. S. Herman, J. A. Lipton, Measuring characteristics of scientific research: A comparison of bibliographic and survey data,Scientometrics, 24 (1992) 359–370.
The citation data collected in 1985–87 and the questionnaire data collected in 1991 were both for the author's dissertation research. Collecting both types of data simultaneously should be more ideal, but the time lapse between the two sets of data should not pose a major threat to validity because overall citation patterns as well as peer judgment usually remain quite stable over a short period of time. For more details on the data collection methods, seeC. Y. K. So,Mapping the Intellectual Landscape of Communication Studies: An Evaluation of Its Disciplinary Status, Unpublished Ph.D. dissertation, Annenberg School for Communication, University of Pennsylvania, 1995 (UMI Number 9615129).
D. Crane, Social structure in a group of scientists: A test of the “Invisible College” hypothesis,American Sociological Review, 34 (1969), 335–352.
E. Garfield,Citation Indexing: Its Theory and Application in Science, Technology, and Humanities, ISI Press, Philadelphia, 1979.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
So, C.Y.K. Citation ranking versus expert judgment in evaluating communication scholars: Effects of research specialty size and individual prominence. Scientometrics 41, 325–333 (1998). https://doi.org/10.1007/BF02459049
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF02459049