Skip to main content
Springer Nature Link
Log in

Unpacking Research Contributions: Investigation from Contextual and Processual Perspectives

  • Conference paper
  • First Online:
Wisdom, Well-Being, Win-Win (iConference 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14597))

Included in the following conference series:

  • 728 Accesses

Abstract

Traditionally, scientific evaluation leaning on quantity and citation metrics rarely places a study within a specific context or a particular historical process for examination, making it difficult to fully reveal its substantial contributions. In a specific scientific field, each study focuses on a certain topic and corresponds to a certain evolutionary stage of the topic. However, few studies analyze research contributions from context-oriented and process-oriented perspectives. This study investigates the contributions of research under several representative topics in the field of quantitative science studies, using articles published in the international journal Scientometrics as samples. BERTopic model is employed for topic clustering, and four research topics are selected for in-depth analysis. In order to unveil research contributions to knowledge production and to different audiences, various metrics including disruptiveness, citation impact and altmetrics are combined for indicator-level analysis, and articles are classified into different categories according to the knowledge contribution types and research orientations for content-level analysis. Results reveal that representative research topics exhibit greater disruptiveness and research impact compared to the overall sample. However, as research topics develop, there is a declining trend in introducing new knowledge and producing impact within academia. Simultaneously, there is a certain degree of enhancement in their impact beyond academia, and also a shift in knowledge contribution types and research orientations. Our findings contribute to a contextual and processual understanding of diverse research contributions, serving as a reference for the evaluation practices of research outcomes oriented towards contribution assessment.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Park, M., Leahey, E., Funk, R.J.: Papers and patents are becoming less disruptive over time. Nature 613(7942), 138–144 (2023)

    Article  Google Scholar 

  2. Zitt, M., Ramanana-Rahary, S., Bassecoulard, E.: Relativity of citation performance and excellence measures: from cross-field to cross-scale effects of field-normalisation. Scientometrics 63(2), 373–401 (2005)

    Article  Google Scholar 

  3. Sugimoto, C.R., Weingart, S.: The kaleidoscope of disciplinarity. Journal of Documentation 71(4), 775–794 (2015)

    Article  Google Scholar 

  4. Waltman L., Van Eck N.J.: Field Normalization of Scientometric Indicators. In: GLäNZEL W, MOED H F, SCHMOCH U, et al. Springer Handbook of Science and Technology Indicators. Cham; Springer International Publishing, pp. 281–300 (2019)

    Google Scholar 

  5. Ivancheva, L.: Scientometrics today: a methodological overview. COLLNET J. Scientometrics Inform. Manage. 2(2), 47–56 (2008)

    Article  Google Scholar 

  6. Carayannis, E.G., Campbell, D.F.J.: Definition of Key Terms: Knowledge, Knowledge Production, Innovation, Democracy, and Governance [M]. In: CARAYANNIS, E.G., CAMPBELL, D.F.J., Smart Quintuple Helix Innovation Systems: How Social Ecology and Environmental Protection are Driving Innovation, Sustainable Development and Economic Growth. Cham; Springer International Publishing, pp. 5–15 (2019)

    Google Scholar 

  7. Kuhn, T.S.: The Structure of Scientific Revolutions, 2nd edn. [M]. Chicago: University of Chicago Press (1970)

    Google Scholar 

  8. Leibel, C., Bornmann, L.: What do we know about the disruption index in scientometrics? An overview of the literature. Scientometrics (2023)

    Google Scholar 

  9. Chao, W., Chen, M., Zhou, X., et al.: A joint framework for identifying the type and arguments of scientific contribution. Scientometrics 128(6), 3347–3376 (2023)

    Article  Google Scholar 

  10. Chen, H., Nguyen, H., Alghamdi, A.: Constructing a high-quality dataset for automated creation of summaries of fundamental contributions of research articles. Scientometrics 127(12), 7061–7075 (2022)

    Article  Google Scholar 

  11. Bernal, J.D.: The social function of science [M]. Hertford: Stephen Austin and Son (1938)

    Google Scholar 

  12. Vakkari, P., Järvelin, K., Chang, Y.-W.: The association of disciplinary background with the evolution of topics and methods in library and information science research 1995–2015. J. Am. Soc. Inf. Sci. 74(7), 811–827 (2023)

    Google Scholar 

  13. Lund, B.D.: Who really contributes to information science research? an analysis of disciplinarity and nationality of contributors to ten top journals. Malays. J. Libr. Inf. Sci. 25(3), 15–29 (2020)

    Google Scholar 

  14. Wu, L., Wang, D., Evans, J.A.: Large teams develop and small teams disrupt science and technology. Nature 566(7744), 378–382 (2019)

    Article  Google Scholar 

  15. Liu, X., Shen, Z., Liao, Y., et al.: The research about the improved disruption index and its influencing factors (in Chinese). Libr. Inform. Serv. 64(24), 84–91 (2020)

    Google Scholar 

  16. Bornmann, L., Tekles, A.: Disruption index depends on length of citation window. Profesional de la información 28(2), e280207 (2019)

    Article  Google Scholar 

  17. Grootendorst, M.: BERTopic: neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:220305794 (2022)

  18. Devlin, J., Chang, M.-W., Lee, K., et al.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:181004805 (2018)

  19. Egger, R., Yu, J.: A topic modeling comparison between LDA, NMF, Top2Vec, and BERTopic to Demystify Twitter Posts. Frontiers in Sociology 7(2022)

    Google Scholar 

  20. Contreras, K., Verbel, G., Sanchez, J., et al.: Using topic modelling for analyzing panamanian parliamentary proceedings with neural and statistical methods. In: proceedings of the 2022 IEEE 40th Central America and Panama Convention (CONCAPAN), F 9–12 Nov. 2022 (2022)

    Google Scholar 

  21. Hristova, G., Netov, N.: Media coverage and public perception of distance learning during the COVID-19 pandemic: a topic modeling approach based on BERTopic. In: Proceedings of the 2022 IEEE International Conference on Big Data (Big Data), F 17–20 Dec 2022 (2022)

    Google Scholar 

  22. Wang, Z., Chen, J., Chen, J., et al.: Identifying interdisciplinary topics and their evolution based on BERTopic. Scientometrics (2023)

    Google Scholar 

  23. Zhang, L., Sivertsen, G., Du, H., et al.: Gender differences in the aims and impacts of research. Scientometrics 126(11), 8861–8886 (2021)

    Article  Google Scholar 

  24. Hirsch, J.E.: An index to quantify an individual’s scientific research output. Proc. Natl. Acad. Sci. 102(46), 16569–16572 (2005)

    Article  Google Scholar 

  25. Fox, M.F., Faver, C.A.: Men, women, and publication productivity: patterns among social work academics. Sociol. Q. 26(4), 537–549 (1985)

    Article  Google Scholar 

  26. Long, J.S.: The origins of sex differences in science. Soc. Forces 68(4), 1297–1316 (1990)

    Article  Google Scholar 

  27. Zuckerman, H., Cole, J.R.: Women in American science. Minerva 13(1), 82–102 (1975)

    Article  Google Scholar 

  28. Ross, M.B., Glennon, B.M., Murciano-Goroff, R., et al.: Women are credited less in science than men. Nature 608(7921), 135–145 (2022)

    Article  Google Scholar 

  29. Priem, J., Hemminger, B.H.: Scientometrics 2.0: New metrics of scholarly impact on the social Web. First Monday 15(7) (2010)

    Google Scholar 

  30. Van Eck, N.J., Waltman, L.: Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics 84(2), 523–538 (2010)

    Article  Google Scholar 

  31. Mchugh, M.L.: Interrater reliability: the kappa statistic. Biochemia medica 22(3), 276–282 (2012)

    Article  MathSciNet  Google Scholar 

  32. Leahey, E., Lee, J., Funk, R.J.: What types of novelty are most disruptive? Am. Sociol. Rev. 88(3), 562–597 (2023)

    Article  Google Scholar 

  33. Egghe, L.: Theory and practise of the g-index. Scientometrics 69(1), 131–152 (2006)

    Article  Google Scholar 

  34. Hirsch, J.E.: Hα: An index to quantify an individual’s scientific leadership. Scientometrics 118(2), 673–686 (2019)

    Article  Google Scholar 

  35. Söderlund, T., Madison, G.: Objectivity and realms of explanation in academic journal articles concerning sex/gender: a comparison of Gender studies and the other social sciences. Scientometrics 112(2), 1093–1109 (2017)

    Article  Google Scholar 

  36. Moed, H.F.: A critical comparative analysis of five world university rankings. Scientometrics 110(2), 967–990 (2017)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (grant no. 71974150, 72374160), and the National Laboratory Center for Library and information Science in Wuhan University.

Author information

Authors and Affiliations

  1. School of Information Management, Wuhan University, Wuhan, China

    Zhe Cao, Lin Zhang & Ying Huang

  2. Center for Science, Technology & Education Assessment (CSTEA), Wuhan University, Wuhan, China

    Zhe Cao, Lin Zhang & Ying Huang

  3. Chinese Academy of Social Sciences Evaluation Studies, Beijing, China

    Yuanyuan Shang

  4. Centre for R&D Monitoring (ECOOM) and Department MSI, KU Leuven, Louvain, Belgium

    Lin Zhang & Ying Huang

Authors
  1. Zhe Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuanyuan Shang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ying Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lin Zhang .

Editor information

Editors and Affiliations

  1. iSchool organization, Berlin, Germany

    Isaac Sserwanga

  2. University of Tsukuba, Tsukuba, Japan

    Hideo Joho

  3. Jilin University, Changchun, China

    Jie Ma

  4. Stockholm University, Kista, Sweden

    Preben Hansen

  5. Wuhan University, Wuhan, China

    Dan Wu

  6. University of Tsukuba, Tsukuba, Japan

    Masanori Koizumi

  7. University of California, Los Angeles, CA, USA

    Anne J. Gilliland

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cao, Z., Shang, Y., Zhang, L., Huang, Y. (2024). Unpacking Research Contributions: Investigation from Contextual and Processual Perspectives. In: Sserwanga, I., et al. Wisdom, Well-Being, Win-Win. iConference 2024. Lecture Notes in Computer Science, vol 14597. Springer, Cham. https://doi.org/10.1007/978-3-031-57860-1_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-57860-1_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-57859-5

  • Online ISBN: 978-3-031-57860-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics