Abstract
In this study we present a method for systematic investigation of the diversity in academic performance and its influence factors among successful scholars. In addition, we examine the potential effect of citation indices on the scholarly performance evaluation. To this end, a quantitative research was conducted on the data of 663 tenured professors, sampled from six faculties in two Israeli universities. The scholars’ productivity and impact rates were collected from the two major citation indices: Web of Science (WoS) and Google Scholar (GS). A comparison was carried out among the highest impact, lowest impact and average impact scholars in the corpus for each citation index. Significant differences were found between scholars’ performance rates in different impact-level groups in the two indices. The top performing group comprised 44 scholars who belonged to the highest impact sub-corpora according to both citation indices. Linear regression analysis showed that women, despite being a minority in the Israeli academia, outperformed men in terms of scientific impact. Interestingly, there were several differences among the two indices in terms of seniority and performance rates. Our findings provide evidence for the “rich get richer” phenomenon in GS compared to WoS. In WoS mean performance rates stabilize after 15 years of seniority, while in GS performance rates of the scholars constantly grow over time. The study contributes to the evaluation of scholarly success and performance diversity in the academic community. The obtained results provide useful insights on academic success and promotion policies for researchers and institutions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abramo, G., D’Angelo, C. A., & Caprasecca, A. (2009a). Gender differences in research productivity: A bibliometric analysis of the Italian academic system. Scientometrics, 79(3), 517–539. https://doi.org/10.1007/s11192-007-2046-8.
Abramo, G., D’Angelo, C. A., & Di Costa, F. (2009b). Research collaboration and productivity: Is there correlation. Higher Education, 57(2), 155–171. https://doi.org/10.1007/s10734-008-9139-z.
Abramo, G., D’Angelo, C. A., & Murgia, G. (2016). The combined effects of age and seniority on research performance of full professors. Science and Public Policy, 43(3), 301–319. https://doi.org/10.1093/scipol/scv037.
Adams, S. J. D., Black, G. C., Clemmons, J. R., Paula, E., & Stephan, P. E. (2005). Scientific teams and institutional collaborations: Evidence from U.S. universities, 1981–1999. Research Policy, 34(3), 259–285. https://doi.org/10.1016/j.respol.2005.01.014.
Adriaanse, L. S., & Rensleigh, C. (2013). Web of Science, Scopus and Google Scholar: A content comprehensiveness comparison. Electronic Library, 31(6), 727–744. https://doi.org/10.1108/EL-12-2011-0174.
Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2009). h-Index: A review focused in its variants, computation and standardization for different scientific fields. Journal of Informetrics, 3(4), 273–289. https://doi.org/10.1016/j.joi.2009.04.001.
Amara, N., & Landry, R. (2012). Counting citations in the field of business and management: Why use Google Scholar rather than the Web of Science. Scientometrics, 93(3), 553–581. https://doi.org/10.1007/s11192-012-0729-2.
Andersen, J. P., & Nielsen, W. (2018). Google Scholar and Web of Science: Examining gender differences in citation coverage across five scientific disciplines. Journal of Informetrics, 12(3), 950–959. https://doi.org/10.1016/j.joi.2018.07.010.
Anker, M. S., Hadzibegovic, S., Lena, A., & Haverkamp, W. (2019). The difference in referencing in Web of Science, Scopus, and Google Scholar. ESC Heart Failure, 6(6), 1291–1312. https://doi.org/10.1002/ehf2.12583.
Bakkalbasi, N., Bauer, K., Glover, J., & Wang, L. (2006). Three options for citation tracking: Google Scholar, Scopus and Web of Science. Biomedical Digital Libraries, 3(7), 1–8. https://doi.org/10.1186/1742-5581-3-7.
Bar-Ilan, J. (2008). Which h-index? A comparison of WoS, Scopus and Google Scholar. Scientometrics, 74(2), 257–271. https://doi.org/10.1007/s11192-008-0216-y.
Bergman, E. M. L. (2012). Finding citations to social work literature: The relative benefits of using Web of Science, Scopus, or Google Scholar. Journal of Academic Librarianship, 38(6), 370–379. https://doi.org/10.1016/j.acalib.2012.08.002.
Blackburn, R. T., Behymer, C. E., & Hall, D. E. (1978). Research note: Correlates of faculty publications. Sociology of Education, 51(2), 132–141. https://doi.org/10.2307/2112245.
Bordons, M., Gómez, I., Fernández, M. T., Zulueta, M. A., & Méndez, A. (1996). Local, domestic and international scientific collaboration in biomedical research. Scientometrics, 37(2), 279–295. https://doi.org/10.1007/BF02093625.
Brizan, D. G., Gallagher, K., Jahangir, A., & Brown, T. (2016). Predicting citation patterns: Defining and determining influence. Scientometrics, 108(1), 183–200. https://doi.org/10.1007/s11192-016-1950-1.
Campbell, P. G., Awe, O. O., Maltenfort, M. G., Moshfeghi, D. M., Leng, T., Moshfeghi, A. A., et al. (2011). Medical school and residency influence on choice of an academic career and academic productivity among neurosurgery faculty in the United States. Clinical article. Journal of Neurosurgery, 115(2), 380–386. https://doi.org/10.3171/2011.3.JNS101176.
Costas, R., & Bordons, M. (2007). The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level. Journal of Informetrics, 1(3), 193–203. https://doi.org/10.1016/j.joi.2007.02.001.
De Groote, S. L., & Raszewski, R. (2012). Coverage of Google Scholar, Scopus, and Web of Science: A case study of the h-index in nursing. Nursing Outlook, 60(6), 391–400. https://doi.org/10.1016/j.outlook.2012.04.007.
De Winter, J. C. F., Zadpoor, A. A., & Dodou, D. (2014). The expansion of Google Scholar versus Web of Science: A longitudinal study. Scientometrics, 98(2), 1547–4565. https://doi.org/10.1007/s11192-013-1089-2.
Ductor, L. (2015). Does co-authorship lead to higher academic productivity? Oxford Bulletin of Economics and Statistics, 77(3), 385–407. https://doi.org/10.1111/obes.12070.
Egghe, L. (2006). Theory and practise of the g-index. Scientometrics, 69(1), 131–152. https://doi.org/10.1007/s11192-006-0144-7.
Eloy, J. A., Svider, P., Chandrasekhar, S. S., Husain, Q., Mauro, K. M., Setzen, M., et al. (2013). Gender disparities in scholarly productivity within academic otolaryngology departments. Otolaryngology-Head and Neck Surgery, 148(2), 215–222. https://doi.org/10.1177/0194599812466055.
Falagas, M. E., Pitsouni, E. I., Malietzis, G. A., & Pappas, G. (2008). Comparison of PubMed, Scopus, Web of Science, and Google Scholar: Strengths and weaknesses. FASEB Journal, 22(2), 338–342. https://doi.org/10.1096/fj.07-9492LSF.
Fenner, T., Harris, M., Levene, M., & Bar-Ilan, J. (2018). A novel bibliometric index with a simple geometric interpretation. PLoS ONE, 13(1), 1–14. https://doi.org/10.1371/journal.pone.0200098.
Franceschet, M. (2010). A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar. Scientometrics, 83(1), 243–258. https://doi.org/10.1007/s11192-009-0021-2.
Fulton, O., & Trow, M. (1974). Research activity in American higher education. Sociology of Education, 47(1), 29–73. https://doi.org/10.2307/2112166.
García-Pérez, M. A. (2010). Accuracy and completeness of publication and citation records in the Web of Science, PsycINFO, and Google Scholar: A case study for the computation of h indices in Psychology. Journal of the American Society for Information Science and Technology, 61(10), 2070–2085. https://doi.org/10.1002/asi.21372.
Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science, 178, 471–479. https://doi.org/10.1126/science.178.4060.471.
Garfield, E. (2009). From the science of science to Scientometrics visualizing the history of science with HistCite software. Journal of Informetrics, 3(3), 173–179. https://doi.org/10.1016/j.joi.2009.03.009.
Halevi, G., Moed, H., & Bar-Ilan, J. (2017). Suitability of Google Scholar as a source of scientific information and as a source of data for scientific evaluation: Review of the literature. Journal of Informetrics, 11(3), 823–834. https://doi.org/10.1016/j.joi.2017.06.005.
Harzing, A. W., & Alakangas, S. (2016). Google Scholar, Scopus and the Web of Science: A longitudinal and cross-disciplinary comparison. Scientometrics, 106(2), 787–804. https://doi.org/10.1007/s11192-015-1798-9.
Hazoglou, M. J., Kulkarni, V., Skiena, S. S., & Dill, K. A. (2017). Citation histories of papers: Sometimes the rich get richer, sometimes they don’t. Retrieved from https://arxiv.org/abs/1703.04746.
Henderson, S. O., & Brestky, P. (2003). Predictors of academic productivity in emergency medicine. Academic Emergency Medicine, 10(9), 1009–1011. https://doi.org/10.1111/j.1553-2712.2003.tb00661.x.
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). The Leiden Manifesto for research metrics. Nature, 520(7548), 429–431. https://doi.org/10.1038/520429a.
Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572. https://doi.org/10.1073/pnas.0507655102.
Hirsch, J. E. (2007). Does the h index have predictive power? Proceedings of the National Academy of Sciences, 104(9), 19193–19198. https://doi.org/10.1073/pnas.0707962104.
Jascó, P. (2005). Google Scholar: The pros and cons. Online Information Review, 29(2), 208–214. https://doi.org/10.1108/1468452051059806.
Ke, W. (2013). A fitness model for scholarly impact analysis. Scientometrics, 94(3), 981–998. https://doi.org/10.1007/s11192-012-0787-5.
Kulkarni, A. V., Aziz, B., & Shams, I. (2009). Comparisons of citations in Web of Science, Scopus, and Google Scholar for articles published in general medical journals. Journal of the American Medical Association (JAMA), 302(10), 1092–1096. https://doi.org/10.1001/jama.2009.1307.
Kyvik, S. (1990). Age and scientific productivity: Differences between fields of learning. Higher Education, 19(1), 37–55. https://doi.org/10.1007/BF00142022.
Kyvik, S., & Teigen, M. (1996). Child care, research collaboration, and gender differences in scientific productivity. Science, Technology and Human Values, 21(1), 54–71. https://doi.org/10.1177/016224399602100103.
Landry, R., Traore, N., & Godin, B. (1996). An econometric analysis of the effect of collaboration on academic research productivity. Higher Education, 32(3), 283–301. https://doi.org/10.1007/BF00138868.
Lariviére, V., Vignola-Gagné, E., Villeneuve, C., Gelinas, P., & Gingras, Y. (2011). Sex differences in research funding, productivity and impact: An analysis of Quebec university professors. Scientometrics, 87(3), 483–498. https://doi.org/10.1007/s11192-011-0369-y.
Levene, M., Fenner, T., & Bar-Ilan, J. (2019). Characterisation of the χ-index and the rec-index. Scientometrics, 120(2), 885–896. https://doi.org/10.1007/s11192-019-03151-7.
Lindahl, J., Colliander, C., & Danell, R. (2020). Early career performance and its correlation with gender and publication output during doctoral education. Scientometrics, 122, 309–330. https://doi.org/10.1007/s11192-019-03262-1.
Martín-Martín, A., Orduna-Malea, E., Thelwall, M., & López-Cózar, E. D. (2018). Google Scholar, Web of Science, and Scopus: A systematic comparison of citations in 252 subject categories. Journal of Informetrics, 12(4), 1160–1177. https://doi.org/10.1016/j.joi.2018.09.002.
Martin-Sempere, M. J., Rey-Rocha, J., & Garzon-Garcia, B. (2002). The effect of team consolidation on research collaboration and performance of scientists. Case study of Spanish university researchers in Geology. Scientometrics, 55(3), 377–394. https://doi.org/10.1023/A:1020462712923.
Meho, L. I., & Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus Scopus and Google Scholar. Journal of the American Society for Information Science and Technology, 58(13), 2105–2125. https://doi.org/10.1002/asi.20677.
Mikki, S. (2010). Comparing Google Scholar and ISI Web of Science for Earth Sciences. Scientometrics, 82(2), 321–331. https://doi.org/10.1007/s11192-009-0038-6.
Mingers, J., & Lipitakis, E. A. E. C. G. (2010). Counting the citations: A comparison of Web of Science and Google Scholar in the field of business and management. Scientometrics, 85(2), 613–625. https://doi.org/10.1007/s11192-010-0270-0.
Missen, M. M. S., Qureshi, S., Salamat, N., Akhtar, N., Asmat, H., Coustaty, M., et al. (2020). Scientometric analysis of social science and science disciplines in a developing nation: A case study of Pakistan in the last decade. Scientometrics. https://doi.org/10.1007/s11192-020-03379-8.
Mitra, P. (2006). Hirsch-type indices for ranking institutions scientific research output. Current Science, 91(11), 1439.
Pagel, P. S., & Hudetz, J. A. (2011). An analysis of scholarly productivity in United States academic anaesthesiologists by citation bibliometrics. Anaesthesia, 66(10), 873–878. https://doi.org/10.1111/j.1365-2044.2011.06860.x.
Pan, R. K., & Fortunato, S. (2014). Author impact factor: Tracking the dynamics of individual scientific impact. Scientific Reports, 4, 4880. https://doi.org/10.1038/srep04880.
Petersen, A. M., Fortunato, S., Pan, R. K., Kaski, K., Penner, O., Rungi, A., et al. (2014). Reputation and impact in academic careers. Proceedings of the National Academy of Sciences of the United States of America, 111(43), 15316–15321. https://doi.org/10.1073/pnas.1323111111.
Petersen, A. M., Riccaboni, M., Stanley, H. E., & Pammolli, F. (2012). Persistence and uncertainty in the academic career. Proceedings of the National Academy of Sciences of the United States of America, 109(14), 5213–5218. https://doi.org/10.1073/pnas.1121429109.
Prins, A. A. M., Costas, R., van Leeuwen, T. N., & Wouters, P. F. (2016). Using Google Scholar in research evaluation of humanities and social science programs: A comparison with Web of Science data. Research Evaluation, 25(3), 264–270. https://doi.org/10.1093/reseval/rvv049.
Pritchard, A. (1969). Statistical bibliography or bibliometrics? Journal of Documentation, 25, 348–349.
Raban, D. R., & Rabin, E. (2007). The power of assuming normality. In Proceedings of the European and Mediterranean Conference on Information Systems 2007. Valencia, Spain: Polytechnic University of Valencia.
Raj, A., Carr, P. L., Kaplan, S. E., Terrin, N., Breeze, J. L., & Freund, K. M. (2016). Longitudinal analysis of gender differences in academic productivity among medical faculty across 24 medical schools in the United States. Academic Medicine, 91(8), 1074–1079. https://doi.org/10.1097/ACM.0000000000001251.
Reed, D. A., Enders, F., Lindor, R., McClees, M., & Lindor, K. D. (2011). Gender differences in academic productivity and leadership appointments of physicians throughout academic careers. Academic Medicine, 86(1), 43–47. https://doi.org/10.1097/ACM.0b013e3181ff9ff2.
Sabharwal, M. (2013). Comparing research productivity across disciplines and career stages. Journal of Comparative Policy Analysis: Research and Practice, 15(2), 141–163. https://doi.org/10.1080/13876988.2013.785149.
Sher, I. H., & Garfield, E. (1965). New tools for improving and evaluating the effectiveness of research. In Research program effectiveness, proceedings of the conference sponsored by the Office of Naval Research, Washington, DC (pp. 135–146).
Simoes, N., & Crespo, N. (2020). A flexible approach for measuring author-level publishing performance. Scientometrics, 122, 331–355. https://doi.org/10.1007/s11192-019-03278-7.
Simon, H. A. (1955). On a class of skew distribution functions. Biometrika, 42(3–4), 425–440. https://doi.org/10.2307/2333389.
Simonton, D. K. (1997). Creative productivity: A predictive and explanatory model of career trajectories and landmarks. Psychological Review, 104(1), 66–89. https://doi.org/10.1037/0033-295X.104.1.66.
Sinatra, R., Wang, D., Deville, P., Song, C., & Barabási, A. L. (2016). Quantifying the evolution of individual scientific impact. Science, 354(6312), 596–604. https://doi.org/10.1126/science.aaf5239.
Stack, S. (2004). Gender, children and research productivity. Research in Higher Education, 45(8), 891–920. https://doi.org/10.1007/s11162-004-5953-z.
Tomei, K. L., Nahass, M. M., Husain, Q., Agrawal, N., Patel, S. K., Svider, P. F., et al. (2014). A gender-based comparison of academic rank and scholarly productivity in academic neurological surgery. Journal of Clinical Neuroscience, 21(7), 1102–1105. https://doi.org/10.1016/j.jocn.2013.11.006.
Waltman, L., & van Eck, N. J. (2012). The inconsistency of the h-index. Journal of the American Society for Information Science and Technology, 63(2), 406–415. https://doi.org/10.1002/asi.21678.
Wanner, R. A., Lewis, L. S., & Gregorio, D. I. (1981). Research productivity in academia: A comparative study of the sciences, social sciences and humanities. Sociology of Education, 54(4), 238–253. https://doi.org/10.2307/2112566.
Webster, G. D., Jonason, P. K., & Schember, T. O. (2009). Hot topics and popular papers in evolutionary psychology: Analyses of title words and citation counts in Evolution and Human Behavior, 1979–2008. Evolutionary Psychology, 7(3), 348–362. https://doi.org/10.1177/147470490900700301.
Weinberger, M., Zhitomirsky-Geffet, M., & Bouhnik, D. (2020). Academic and demographic characteristics as predictors of scholarly productivity in the Israeli academia. Poster presented at iConference 2020, Borås, Sweden.
Weingart, P. (2005). Impact of bibliometrics upon the science system: Inadvertent consequences? Scientometrics, 62(1), 117–131. https://doi.org/10.1007/s11192-005-0007-7.
Wildgaard, L. (2015). A comparison of 17 author-level bibliometric indicators for researchers in astronomy, Environmental Science, Philosophy and Public Health in Web of Science and Google Scholar. Scientometrics, 104(3), 873–906. https://doi.org/10.1007/s11192-015-1608-4.
Yair, G., Gueta, N., & Davidovitch, N. (2017). The law of limited excellence: publication productivity of Israel Prize laureates in the life and exact sciences. Scientometrics, 113, 299–311. https://doi.org/10.1007/s11192-017-2465-0.
Yang, G., Villalta, J. D., Weiss, D. A., Carroll, P. R., & Breyer, B. N. (2012). Gender differences in academic productivity and academic career choice among urology residents. Journal of Urology, 188(4), 1286–1290. https://doi.org/10.1016/j.juro.2012.06.022.
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
See Tables 6, 7, 8, 9, 10, 11 and Figs. 11, 12, 13.
The total publication rate distribution of the common scholars in the two citation indices
The total citation rate distribution of the common scholars in the two citation indices
The total h-index rate distribution of the common scholars in the two citation indices
Rights and permissions
About this article
Cite this article
Weinberger, M., Zhitomirsky-Geffet, M. Diversity of success: measuring the scholarly performance diversity of tenured professors in the Israeli academia. Scientometrics 126, 2931–2970 (2021). https://doi.org/10.1007/s11192-020-03823-9
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-020-03823-9