Abstract
We compare a new method for measuring research leadership with the traditional method. Both methods are objective and reliable, utilize standard citation databases, and are easily replicated. The traditional method uses partitions of science based on journal categories, and has been extensively used to measure national leadership patterns in science, including those appearing in the NSF Science & Engineering Indicators Reports and in prominent journals such as Science and Nature. Our new method is based on co-citation techniques at the paper level. It was developed with the specific intent of measuring research leadership at a university, and was then extended to examine national patterns of research leadership. A comparison of these two methods provides compelling evidence that the traditional method grossly underestimates research leadership in most countries. The new method more accurately portrays the actual patterns of research leadership at the national level.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The list of journals and categories is available from Lawrence Burton, NSF/SRS.
Thomson Reuters does have a separate Proceedings database.
We were concerned that the new journal classification system would generate some categories that would be so small as to be insignificant. For this reason, we set a minimum category size equal in size to the smallest NSF category, General Engineering, which covers about 350 articles annually. Thirty-four of the STS categories were smaller than this value, and are not included in the analysis.
References
Bassecoulard, E., & Zitt, M. (1999). Indicators in a research institute: A multi-level classification of journals. Scientometrics, 44(3), 323–345.
Boyack, K. W. (2009). Using detailed maps of science to identify potential collaborations. Scientometrics, 79(1), 27–44.
Calero-Medina, C., Lopez-Illescas, C., Visser, M. S., & Moed, H. F. (2008). Important factors when interpreting bibliometric rankings of world universities: an example from oncology. Research Evaluation, 17(1), 71–81.
Franklin, J. J., & Johnston, R. (1988). Co-citation bibliometric modeling as a tool for S&T policy and R&D management: Issues, applications, and developments. In A. F. J. van Raan (Ed.), Handbook of quantitative studies of science and technology (pp. 325–389). North-Holland, BV: Elsevier Science Publishers.
Glänzel, W., DeBackere, K., & Meyer, M. (2008). ‘Triad’ or ‘tetrad’? On global changes in a dynamic world. Scientometrics, 74(1), 71–88.
Glänzel, W., & Schubert, A. (2003). A new classification scheme of science fields and subfields designed for scientometric evaluation purposes. Scientometrics, 56(3), 357–367.
Glänzel, W., Thijs, B., Schubert, A., & DeBackere, K. (2009). Subfield-specific normalized relative indicators and a new generation of relational charts: Methodological foundations illustrated on the assessment of institutional research performance. Scientometrics, 78(1), 165–188.
Hill, D., Rapoport, A. I., Lehming, R. F., & Bell, R. K. (2007). Changing U.S. output of scientific articles: 1988–2003 (No. NSF 07–320). Arlington, VA: National Science Foundation, Division of Science Resource Statistics.
King, D. A. (2004). The scientific impact of nations. Nature, 430, 311.
Klavans, R., & Boyack, K. W. (2006). Quantitative evaluation of large maps of science. Scientometrics, 68(3), 475–499.
Klavans, R., & Boyack, K. W. (2007). Is there a convergent structure of science? A comparison of maps using the ISI and Scopus databases. Paper presented at the 11th International Conference of the International Society for Scientometrics and Informetrics.
Klavans, R., & Boyack, K. W. (2008). Thought leadership: A new indicator for national and institutional comparison. Scientometrics, 75(2), 239–250.
Klavans, R., & Boyack, K. W. (2009). Toward a consensus map of science. Journal of the American Society for Information Science and Technology, 60(3), 455–476.
Leydesdorff, L., & Wagner, C. (2009). Is the United States losing ground in science? A global perspective on the world science system. Scientometrics, 78(1), 23–26.
Martin, S., Brown, W. M., Klavans, R., & Boyack, K. W. (2008). DrL: Distributed recursive (graph) layout, SAND2008-2936J.
May, R. M. (1997). The scientific wealth of nations. Science, 275, 793–796.
National_Science_Board. (2008). Science and engineering indicators 2008 (Vol. 1, NSB 08-01; Vol. 2, NSB 08-01A). Arlington, VA: National Science Foundation.
Small, H. (1999). Visualizing science by citation mapping. Journal of the American Society for Information Science, 50(9), 799–813.
Small, H., & Griffith, B. C. (1974). The structure of scientific literatures, I: Identifying and graphing specialties. Social Studies of Science, 4, 17–40.
Small, H., Sweeney, E., & Greenlee, E. (1985). Clustering the Science Citation Index using co-citations. II. Mapping science. Scientometrics, 8(5–6), 321–340.
Acknowledgements
Funding for development of the STS journal classification schema and evaluation of research leadership at UCSD was provided by the office of the Vice Provost of Research at UCSD.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Klavans, R., Boyack, K.W. Toward an objective, reliable and accurate method for measuring research leadership. Scientometrics 82, 539–553 (2010). https://doi.org/10.1007/s11192-010-0188-6
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-010-0188-6