Skip to main content
SpringerLink
Log in

Using citation counts as a measure of quality in science measuring what's measurable rather than what's valid

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Empirical work in the social studies of science has progressed rapidly with the availability and development of the citation indexes. Citation counts have become a widely accepted measure of the quality of a scientific contribution. However, there are several problems involved in the use of citation counts as a measure of quality in science. First, citation counts are sensitive to popular trends in science. In this sense, they approximate a Nielsen rating for science. Second, the distribution of citations restricts their utility to separating the extremes. Third, citation counts are not sensitive to the ethical and moral dimensions of the quality of a scientific contribution. Fourth, citation counts underestimate the contribution of applied scientists. This paper examines these limitations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. J. G. MANIS, Some academic influences upon publication productivity,Social Forces, 29 (1951) 267.

    Google Scholar 

  2. D. LINDSEY,The Scientific Publication System in Social Science, San Francisco, Jossey-Bass, 1978.

    Google Scholar 

  3. B. T. SHAW,The Use of Quality and Quantity of Publications as, Criteria for Evaluating Scientists, Agriculture Research Service, USDA Miscellaneous Publication No. 1041, Washington, D. C., U.S. Department of Agriculture, 1967.

    Google Scholar 

  4. N. D. GLENN, W. VILLEMEZ, The productivity of sociologists in 45 American universities.American Sociologist, 5 (1970) 244.

    Google Scholar 

  5. L. L. HARGENS,Patterns of Scientific Research: A Comparative Analysis of Research in Three Scientific Fields, Washington, D. C., The American Sociological Association, 1975, p. 87.

    Google Scholar 

  6. H. W. MENARD,Science: Growth and Change, New York, Harper and Row, 1971.

    Google Scholar 

  7. D. CRANE,Invisible Colleges: Diffusion of Knowledge in Scientific Communities, Chicago, University of Chicago Press, 1972.

    Google Scholar 

  8. N. C. MULLINS, The development of specialties in social science: the case of ethnomethodology,Science Studies, 3 (1973) 245.

    Google Scholar 

  9. N. C. MULLINS, New causal theory: An elite specialty in social sciences,History of Political Economy, 7 (1975) 499.

    Google Scholar 

  10. S. COLE, The growth of scientific knowledge: theories of deviance as a case study, In:The Idea of Social Structure, L. COSER (Ed.), New York, Harcourt, Brace, Jovanovich, 1976.

    Google Scholar 

  11. L. FREESE, Cumulative sociological knowledge,American Sociological Review, 37 (1972) 472.

    Google Scholar 

  12. T. PARSONS,Essays in Sociological Theory, (re. ed.) New York, Free Press, 1954.

    Google Scholar 

  13. G. J. STIGLER, The pattern of citation practices in economics. inThe Economist as Preacher and Other Essays, 173–191, Chicago, The University of Chicago Press, 1982.

    Google Scholar 

  14. D. LINDSEY, Assessing precision in the manuscript review process: a little better than a dice roll,Scientometrics, 14 (1988) 61.

    Google Scholar 

  15. H. F. MOED, W. J. M. BURGER, J. G. FRANKFORT, A. F. J. Van RAAN, The use of bibliometric data for measurement of university research performanceResearch Policy, 14 (1985) 131.

    Google Scholar 

  16. R. E. QUANDT, Some quantitative aspects of the economics journal literature.Journal of Political Economy, 84 (1976) 741.

    Google Scholar 

  17. H. A. ZUCKERMAN,The Scientific Elite, New York, Free Press, 1977.

    Google Scholar 

  18. E. GARFIELD,Citation Indexing-its Theory and Applications in Science, Technology, and Humanities, New York, Wiley, 1979.

    Google Scholar 

  19. A. J. DIAMOND, JR. What is a citation worth?The Journal of Human Resources, 21 (1986) 200.

    Google Scholar 

  20. J. COLE, S. COLE,Social Stratification in Science, Chicago, University of Chicago Press, 1973, p. 21.

    Google Scholar 

  21. M. KOCHEN, B. PERKEL, Improving referee-selection and manuscript evaluation, in:Proceedings of the First International Conference of Scientific Editors, J. McCARTNEY (Ed.), April 24–29, 1977, Jerusalem, Dordrecht: Reidel, 1977, p. 1.

    Google Scholar 

  22. E. GARFIELD, The most-cited life science articles highlight AIDS research,Current Contents, 49 (1986) 3.

    Google Scholar 

  23. M. J. MAHONEY, Open exchange and epistemic progress,American Psychologist, 40 (1985) 29.

    Google Scholar 

  24. E. GARFIELD, The most-cited 1983 chemistry articles focus on NMR but the increasing impact of materials science and computer methods is evident,Current Contents, 51-52 (1986) 3.

    Google Scholar 

  25. T. KUHN,The Structure of Scientific Revolutions, Chicago, University of Chicago Press, 1962.

    Google Scholar 

  26. G. HOLTON, Can science be measured? In:Toward a Metric of Science, Y. ELKANA and others (Eds), New York, Wiley-Interscience, 1978.

    Google Scholar 

  27. B. GRIFFITH, H. G. SMALL,A Philadelphia study of structure of science: The structure of social and behavioral science literature, Proceedings of the 1st International Conference on Social Studies of Science, Ithaca, New York November 4–6, 1976.

  28. E. GARFIELD, The 250 most-cited primary authors 1961–1975. Part 3: Each authors's most-cited publication,Current Contents, 51 (1977) 5.

    Google Scholar 

  29. J. WIENER, The footnote fetish,Telos, 31 (1977) 172, p. 174.

    Google Scholar 

  30. H. M. BEVILLE, Jr.Audience Ratings: Radio, Television and Cable, Hillsdale, N. J., Lawrence Erlbaum Associates, Publisher, 1985.

    Google Scholar 

  31. A. C. NIELSEN,What Do the Ratings Really Say? Chicago, A. C. Nielsen, 1964.

    Google Scholar 

  32. L. MEYERS, On the reliability of the ratings,Television Quarterly, 1 (1962) 50.

    Google Scholar 

  33. M. H. MACROBERTS, B. R. MACROBERTS, Quantitative measures of communication in science: A study of the formal level,Social Studies of Science, 16 (1986) 151.

    Google Scholar 

  34. D. DIEKS, H. CHANG, Differences in impact of scientific publications: Some indices derived from a citation analysis.Social Studies of Science, 6 (1976) 247.

    Google Scholar 

  35. M. J. MORAVCSIK, P. MURUGESAN, Some results on the function and quality of citations,Social Studies of Science, 5 (1975) 86.

    Google Scholar 

  36. M. J. MULKAY, Norms and ideology in science,Social Science Information, 15 (1976) 637.

    Google Scholar 

  37. R. A. ROTHMAN, A dissenting view on the scientific ethos,British Journal of Sociology, 23 (1972) 102.

    Google Scholar 

  38. J. LAW, Theories and methods in the sociology of science: an interpretive approach,Social Science Information 13 (1974) 163.

    Google Scholar 

  39. M. JAY,The Dialectical Imagination: A History of the Frankfurt School and the Institute of Social Research, 1923–50, Boston, Little, Brown, 1973.

    Google Scholar 

  40. F. CUNNINGHAM,Objectivity in Social Science, Toronto, University of Toronto Press, 1973.

    Google Scholar 

  41. R. POOLE,Toward Deep Subjectivity, Middlesex, England, Penguin Books, 1972., p. 12.

    Google Scholar 

  42. R. S. LECKER, Whose side are we on?Social Problems, 14 (1967) 239.

    Google Scholar 

  43. R. S. LYNN,Knowledge for What? Boston, Grove Press, 1938.

    Google Scholar 

  44. M. POLANYI,Personal Knowledge, Chicago, University of Chicago Press, 1958.

    Google Scholar 

  45. E. GARFIELD, The 250 most-cited autors in the Arts and Humanities Citation Index, 1976–1983,Current Contents 48 (1986) 3.

    Google Scholar 

  46. M. J. MAHONEY, Publication prejudices: An experimental study of confirmatory bias in the peer review system,Cognitive Therapy and Research, 1 (1977) 161.

    Google Scholar 

  47. T. S. KUHN,The Structure of Scientific Revolutions, Chicago, University of Chicago Press, 1962.

    Google Scholar 

  48. B. F. RESKIN, Sex differences in status attainment in science: the case of the postdoctoral fellowship,American Sociological Review, 42 (1977) 491.

    Google Scholar 

  49. M. H. MACROBERTS, B. R. MACROBERTS, Testing the Ortega Hypothesis: Facts and artifacts,Scientometrics, 12 (1987) 293. Also see the comments and discussions of others regarding this paper in this same issue.

    Google Scholar 

  50. A. V. CICOUREL,Method and Measurement in Sociology, New York, Free Press, 1964.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Human Development and Performance, University of Oregon, (USA)

    D. Lindsey

Authors
  1. D. Lindsey
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lindsey, D. Using citation counts as a measure of quality in science measuring what's measurable rather than what's valid. Scientometrics 15, 189–203 (1989). https://doi.org/10.1007/BF02017198

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02017198

Keywords

Search

Navigation