Abstract
In today’s publish or perish environment, the scholarly impact of a research article holds great importance. The present study examined 2343 articles having both altmetric attention scores and citations published by 22 core health care policy faculty members at Harvard Medical School. Web of Science was used to retrieve the citations, whereas Altmetric Explorer was used to determine the altmetric attention score. The evaluation metrics in this study were focused on article-level information to determine each faculty member’s contribution to the health care policy department collected in November 2018. This paper further determined the predictive capability of altmetric attention score for the increase of citations from November 2018 to January 2020. The findings showed J. Michael McWilliams (Professor); Anupam B. Jena (Associate Professor); and Zirui Song (Assistant Professor) were the faculty members with highest altmetric attention score while Ronald C. Kessler (Professor); Nicole Maestas (Associate Professor); and Zirui Song (Assistant Professor) were the faculty members with the highest citation value. The lowest percentage (5.84%) of articles without any citations were published by the professors, whereas the lowest percentage (4.84%) of articles without any social media mention were published by the assistant professors. The health care faculty had the highest percentage of Mendeley readerships followed by tweets, news, and blogs mention in comparison to a meager percentage of policy-related documents mention for their retrieved articles. A significant strong positive correlation (\(r_{\text{R}}\) > 0.4) was observed between the aggregated ranked altmetric attention scores and ranked citation/increased citation values for all the faculty members. This study can be extended to determine the research productivity of any department, discipline, or group of faculty members.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Akoglu, H. (2018). User’s guide to correlation coefficients. Turkish Journal of Emergency Medicine,18(3), 91–93. https://doi.org/10.1016/j.tjem.2018.08.001.
Alhoori, H., Choudhury, S. R., Kanan, T., Tech, V., Fox, E. A., Tech, V., et al. (2015). On the relationship between open access and altmetrics. https://www.ideals.illinois.edu/bitstream/handle/2142/73451/212_ready.pdf?sequence=2. Retrieved February 3, 2020.
Altmetric. (2020). https://www.altmetric.com/. Retrieved February 2, 2020.
Antelman, K. (2004). Do open-access articles have a greater research impact? College & Research Libraries,65(5), 372–382. https://doi.org/10.5860/crl.65.5.372.
Aoun, S. G., Bendok, B. R., Rahme, R. J., Dacey, R. G., & Batjer, H. H. (2013). Standardizing the evaluation of scientific and academic performance in neurosurgery—Critical review of the “h” index and its variants. World Neurosurgery,80(5), e85–e90.
Azer, S. A., & Azer, S. (2019). Top-cited articles in medical professionalism: A bibliometric analysis versus altmetric scores. British Medical Journal Open,9(7), e029433. https://doi.org/10.1136/bmjopen-2019-029433.
Bornmann, L., & Haunschild, R. (2016). How to normalize Twitter counts? A first attempt based on journals in the Twitter Index. Scientometrics,107(3), 1405–1422. https://doi.org/10.1007/s11192-016-1893-6.
Choi, M., Holliday, E. B., Jagsi, R., Wilson, L. D., Fuller, C. D., & Thomas, C. R. (2014). Citation-based estimation of scholarly activity among domestic academic radiation oncologists: Five-year update. Journal of Radiation Oncology,3(1), 115–122.
Costello, L. C. (2010). Perspective: Is NIH funding the “best science by the best scientists”? A critique of the NIH R01 research grant review policies. Academic Medicine,85(5), 775–779.
Danthi, N., Wu, C. O., Shi, P., et al. (2014). Percentile ranking and citation impact of a large cohort of National Heart, Lung, and Blood Institute-funded cardiovascular R01 grants. Circulation Research,114(4), 600–606.
Harnad, S., & Brody, T. (2004). Comparing the impact of open access vs. non OA articles in the same journals. D-Lib Magazine, 10(6). http://www.dlib.org/dlib/june04/harnad/06harnad.html. Retrieved February 3, 2020.
Hassan, S.-U., Imran, M., Gillani, U., Aljohani, N. R., Bowman, T. D., & Didegah, F. (2017). Measuring social media activity of scientific literature: An exhaustive comparison of scopus and novel altmetrics big data. Scientometrics,113(2), 1037–1057. https://doi.org/10.1007/s11192-017-2512-x.
Haunschild, R., & Bornmann, L. (2016). Normalization of Mendeley reader counts for impact assessment. Journal of Informatics,10(1), 62–73. https://doi.org/10.1016/j.joi.2015.11.003.
Haunschild, R., & Bornmann, L. (2017). How many scientific papers are mentioned in policy-related documents? An empirical investigation using Web of Science and Altmetric data. Scientometrics,110(3), 1209–1216. https://doi.org/10.1007/s11192-016-2237-2.
Health Care Policy. (2020). Harvard University. https://healthcarepolicy.hms.harvard.edu/about. Retrieved February 2, 2020.
Heydari, S., Shekofteh, M., & Kazerani, M. (2019). Relationship between altmetrics and citations a study on the highly cited research papers. DESIDOC Journal of Library & Information Technology,39(4), 169–174. https://doi.org/10.14429/djlit.39.4.14204.
Holmberg, K., & Park, H. W. (2018). An altmetric investigation of the online visibility of South Korea-based scientific journals. Scientometrics,117(1), 603–613. https://doi.org/10.1007/s11192-018-2874-8.
Huang, W., Wang, P., & Wu, Q. (2018). A correlation comparison between Altmetric Attention Scores and citations for six PLOS journals. PLoS ONE,13(4), e0194962. https://doi.org/10.1371/journal.pone.0194962.
Kousha, K., & Abdoli, M. (2010). The citation impact of open access agricultural research: A comparison between OA and non-OA publications. Online Information Review,34(5), 772–785. https://doi.org/10.1108/14684521011084618.
Lawrence, S. (2001). Online or invisible? Nature,411(6837), 521.
Livas, C., & Delli, K. (2018). Looking beyond traditional metrics in orthodontics: An altmetric study on the most discussed articles on the web. European Journal of Orthodontics,40(2), 193–199. https://doi.org/10.1093/ejo/cjx050.
Maggio, L. A., Leroux, T., Meyer, H. S., & Artino, A. R. (2018). Exploring the relationship between altmetrics and traditional measures of dissemination in health professions education. bioRxiv. https://doi.org/10.1101/260059.
Merton, R. K. (1968). The matthew effect in science: The reward and communication systems of science are considered. Science,159(3810), 56–63. https://doi.org/10.1126/science.159.3810.56.
Ortega, J. L. (2018). Reliability and accuracy of altmetric providers: A comparison among Altmetric.com, PlumX and Crossref Event Data. Scientometrics,116(3), 2123–2138. https://doi.org/10.1007/s11192-018-2838-z.
Salajegheh, M., & Dayari, S. (2019). Comparing the citations counts and altmetrics of the top medical science journals in scopus. International Journal of Information Science and Management,17(1), 59–72.
Shu, F., Quan, W., Chen, B., Qiu, J., Sugimoto, C. R., & Larivière, V. (2020). The role of web of science publications in China’s tenure system. Scientometrics. https://doi.org/10.1007/s11192-019-03339-x.
Susarla, H. K., Dhar, V., Karimbux, N. Y., & Tinanoff, N. (2017). Do standard bibliometric measures correlate with academic rank of full-time pediatric dentistry faculty members? Journal of Dental Education,81(4), 427–432.
Syamili, C., & Rekha, R. V. (2017). Do altmetric correlate with citation? A study based on PLOS ONE journal. COLLNET Journal of Scientometrics and Information Management,11(1), 103–117. https://doi.org/10.1080/09737766.2016.1260815.
Thelwall, M. (2018). Using altmetrics to support research evaluation. In M. Erdt, A. Sesagiri Raamkumar, E. Rasmussen, & Y.-L. Theng (Eds.), Altmetrics for research outputs measurement and scholarly information management (pp. 11–28). Singapore: Springer.
Web of Science. (2020). Clarivate analytics. http://login.webofknowledge.com/error/Error?Error=IPError&PathInfo=%2F&RouterURL=http%3A%2F%2Fwww.webofknowledge.com%2F&Domain=.webofknowledge.com&Src=IP&Alias=WOK5. Retrieved February 2, 2020.
Zhang, X., Wang, X., Zhao, H., Ordóñez de Pablos, P., Sun, Y., & Xiong, H. (2019). An effectiveness analysis of altmetrics indices for different levels of artificial intelligence publications. Scientometrics,119(3), 1311–1344. https://doi.org/10.1007/s11192-019-03088-x.
Acknowledgements
The author wishes to thank Altmetric for providing this study’s data free of charge for the research purpose.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Appendix 1: sources and their weights in calculating altmetric attention score
Appendix 1: sources and their weights in calculating altmetric attention score
Sources | Weight |
|---|---|
News | 8 |
Blogs | 5 |
Wikipedia pages | 3 |
Policy documents (per-source) | 3 |
1 | |
Sina Weibo | 1 |
F1000/Publons/Pubpeer | 1 |
LinkedIna,b | 0.5 |
Q&Aa | 0.25 |
Facebook (public pages)a | 0.25 |
Video/YouTubea | 0.25 |
Reddit/Pinteresta,b | 0.25 |
Open Syllabus | 1 |
Google+ | 1 |
Patents | 3 |
Rights and permissions
About this article
Cite this article
Lamba, M. Research productivity of health care policy faculty: a cohort study of Harvard Medical School. Scientometrics 124, 107–130 (2020). https://doi.org/10.1007/s11192-020-03433-5
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-020-03433-5