Abstract
In the last two decades, scholars have designed various types of bibliographic-related indicators to identify breakthrough-class academic achievements. In this study, we take a step further to look at the performance of the promising disruptive index (DI) in reference (Wu et al. in Nature 566(7744):378-382, https://doi.org/10.1038/s41586-019-0941-9, 2019), thus deepening our understanding of the DI and further facilitating its wise use in bibliometrics. Using publication records for Nobel laureates between 1900 and 2016, we calculate the DI of Nobel Prize-winning articles and benchmark articles from each year, use the median and mean DI to denote the central tendency in each year, and analyze the variation of the DI since publication. We find that Nobel Prize-winning articles are not necessarily more disruptive than benchmark articles. Results based on DI depend on the length of their citation time window, and different citation time windows may cause different, even controversial, results. As a result, research assessment should balance between short- & long-term scientific impact; Also, discipline and time play a role in the length of the citation window when using DI to measure the innovativeness of scientific work. The study also discusses potential research directions around DI.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bornmann, L., & Tekles, A. (2019). Disruption index depends on length of citation window. El Profesional De La Información, 28(2), e280207.
Bornmann, L., Tekles, A., Zhang, H. H., & Ye, F. Y. (2019). Do we measure novelty when we analyze unusual combinations of cited references? A validation study of bibliometric novelty indicators based on F1000Prime data. Journal of Informetrics. https://doi.org/10.1016/j.joi.2019.100979
Bu, Y., Waltman, L., & Huang, Y. (2021). A multidimensional framework for characterizing the citation impact of scientific publications. Quantitative Science Studies, 2(1), 155–183. https://doi.org/10.1162/qss_a_00109
Foster, J. G., Rzhetsky, A., & Evans, J. A. (2015). Tradition and innovation in scientists’ research strategies. American Sociological Review, 80(5), 875–908. https://doi.org/10.1177/0003122415601618
Funk, R. J., & Owen-Smith, J. (2017). A dynamic network measure of technological change. Management Science, 63(3), 791–817. https://doi.org/10.1287/mnsc.2015.2366
Garfield, E. (1972). Citation analysis as a tool in journal evaluation: Journals can be ranked by frequency and impact of citations for science policy studies. Science, 178(4060), 471–479.
Garfield, E. (1977). Scientist uses citation indexing to predict winners of Nobel-Prize. Texas Medicine, 73(6), 87–88.
Guo, X., Li, X., & Yu, Y. (2021). Publication delay adjusted impact factor: The effect of publication delay of articles on journal impact factor. Journal of Informetrics, 15(1), 101100. https://doi.org/10.1016/j.joi.2020.101100
Kong, X., Zhang, J., Zhang, D., Bu, Y., Ding, Y., & Xia, F. (2020). The gene of scientific success. ACM Transactions on Knowledge Discovery from Data, 14(4), 1–19. https://doi.org/10.1145/3385530
Kuhn, T. S. (1977). The essential tension: Selected studies in scientific tradition and change. University of Chicago Press.
Li, J., Yin, Y., Fortunato, S., & Wang, D. (2019). A dataset of publication records for Nobel laureates. Scientific Data, 6(1), 33. https://doi.org/10.1038/s41597-019-0033-6
Liang, G., Hou, H., Chen, Q., & Hu, Z. (2020a). Diffusion and adoption: An explanatory model of “question mark” and “rising star” articles. Scientometrics, 124(1), 219–232. https://doi.org/10.1007/s11192-020-03478-6
Liang, G., Hou, H., Ding, Y., & Hu, Z. (2020b). Knowledge recency to the birth of Nobel Prize-winning articles: Gender, career stage, and country. Journal of Informetrics, 14(3), 1–14. https://doi.org/10.1016/j.joi.2020.101053
Liang, G., Hou, H., Hu, Z., Huang, F., Wang, Y., & Zhang, S. (2017). Usage count: A new indicator to detect research fronts. Journal of Data and Information Science, 2(1), 89–104. https://doi.org/10.1515/jdis-2017-0005
Liang, G., Jiang, Y., & Hou, H. (2020c). Same data may bring conflict results: a caution to use the disruptive index. Preprint at http://arXiv.org/2009.06888
Liu Xiaohui, S. Z., Yu, L., & Liying, Y. (2020). The research about the improved disruption index and its influencing factors. Library and Information Service, 64(24), 84–91.
Liu Xiaohui, S. Z., Yu, L., Manman, Z., & Liying, Y. (2021). Research on the stable time window of disruption index. Library and Information Service, 65(18), 49–57.
Park, M., Leahey, E., & Funk, R. J. (2021). The decline of disruption in science and technology. Preprint at http://arXiv.org/2106.11184v4
Purkayastha, A., Palmaro, E., Falk-Krzesinski, H. J., & Baas, J. (2019). Comparison of two article-level, field-independent citation metrics: Field-weighted citation impact (FWCI) and relative citation Ratio (RCR). Journal of Informetrics, 13(2), 635–642. https://doi.org/10.1016/j.joi.2019.03.012
Uzzi, B., Mukherjee, S., Stringer, M., & Jones, B. (2013). Atypical combinations and scientific impact. Science, 342(6157), 468–472. https://doi.org/10.1126/science.1240474
Wang, J., Veugelers, R., & Stephan, P. (2017). Bias against novelty in science: A cautionary tale for users of bibliometric indicators. Research Policy, 46(8), 1416–1436. https://doi.org/10.1016/j.respol.2017.06.006
Wu, L., Wang, D., & Evans, J. A. (2019). Large teams develop and small teams disrupt science and technology. Nature, 566(7744), 378–382. https://doi.org/10.1038/s41586-019-0941-9
Zhang, X., Wang, X., Zhao, H., Ordóñez de Pablos, P., Sun, Y., & Xiong, H. (2019). An effectiveness analysis of altmetrics indices for different levels of artificial intelligence publications. Scientometrics, 119(3), 1311–1344. https://doi.org/10.1007/s11192-019-03088-x
Acknowledgements
The research is an extension based on the previous work of preprint arxiv.org/abs/2009.06888 by the authors. The authors would like to thank Yi Bu and the anonymous reviewers for their thoughtful comments and efforts in improving our manuscript.
Funding
This work was supported by the 2022 Economic and Social Development Issues in Liaoning Province (Project No. 2022lslybkt-034), and the 2021 High-level Technology Innovation Think Tank Youth Project (Project No. 2021ZZZLFZB1207016).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflicts of interest regarding the publication of this paper.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Liang, G., Lou, Y. & Hou, H. Revisiting the disruptive index: evidence from the Nobel Prize-winning articles. Scientometrics 127, 5721–5730 (2022). https://doi.org/10.1007/s11192-022-04499-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-022-04499-z