Abstract
The scholarly social network ResearchGate (RG) started promoting a new composite indicator: Research Interest. That score is built upon a weighting scheme of citations, recommendations, full-text reads, and other reads by RG members. Here I show that the indicator suffers from, at least, two significant issues, which undermine its utility as an alternative metric. Lack of transparency is the former since there are clues that its underlying algorithm is more complex than what is known according to the information conveyed by RG. Redundancy is another issue because the score is essentially driven by the number of citations and seems to bring negligible additional knowledge in the altmetrics landscape.
This is a preview of subscription content, log in via an institution to check access.
References
Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723. https://doi.org/10.1109/TAC.1974.1100705.
Akaike, H. (2011). Akaike’s information criterion. In M. Lovric (Ed.), International encyclopedia of statistical science (pp. 25–25). Berlin, Heidelberg: Springer. https://doi.org/10.1007/978-3-642-04898-2_110.
Banshal, S. K., Singh, V. K., Kaderye, G., Muhuri, P. K., & Sánchez, B. P. (2018). An altmetric analysis of scholarly articles from India. Journal of Intelligent & Fuzzy Systems, 34(5), 3111–3118. https://doi.org/10.3233/JIFS-169495.
Bar-Ilan, J., Haustein, S., Milojević, S., Peters, I., & Wolfram, D. (2018). Peer review, bibliometrics and altmetrics—Do we need them all? Proceedings of the Association for Information Science and Technology, 55(1), 653–656. https://doi.org/10.1002/pra2.2018.14505501073.
Bornmann, L., Haunschild, R., & Adams, J. (2019). Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (REF). Journal of Informetrics, 13(1), 325–340. https://doi.org/10.1016/j.joi.2019.01.008.
Borrego, Á. (2017). Institutional repositories versus ResearchGate: The depositing habits of Spanish researchers. Learned Publishing, 30(3), 185–192. https://doi.org/10.1002/leap.1099.
Copiello, S., & Bonifaci, P. (2018). A few remarks on ResearchGate score and academic reputation. Scientometrics, 114(1), 301–306. https://doi.org/10.1007/s11192-017-2582-9.
Copiello, S., & Bonifaci, P. (2019). ResearchGate score, full-text research items, and full-text reads: A follow-up study. Scientometrics, 119(2), 1255–1262. https://doi.org/10.1007/s11192-019-03063-6.
Deng, S., Tong, J., & Fu, S. (2018). Interaction on an academic social networking sites: A study of ResearchGate Q&A on library and information science. In Proceedings of the 18th ACM/IEEE on joint conference on digital libraries—JCDL’18 (pp. 25–28). New York, New York, USA: ACM Press. https://doi.org/10.1145/3197026.3197062.
Djonov, E., & Van Leeuwen, T. (2018). Social media as semiotic technology and social practice: The case of ResearchGate’s design and its potential to transform social practice. Social Semiotics, 28(5), 641–664. https://doi.org/10.1080/10350330.2018.1504715.
Einhorn, H. J., & Hogarth, R. M. (1975). Unit weighting schemes for decision making. Organizational Behavior and Human Performance, 13(2), 171–192. https://doi.org/10.1016/0030-5073(75)90044-6.
Erdt, M., Nagarajan, A., Sin, S. C. J., & Theng, Y. L. (2016). Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media. Scientometrics, 109(2), 1117–1166. https://doi.org/10.1007/s11192-016-2077-0.
Everitt, B. S., & Skrondal, A. (2010). The Cambridge dictionary of statistics (4th ed.). Cambridge: Cambridge University Press.
Galligan, F., & Dyas-Correia, S. (2013). Altmetrics: Rethinking the way we measure. Serials Review, 39(1), 56–61. https://doi.org/10.1080/00987913.2013.10765486.
González-Valiente, C. L., Pacheco-Mendoza, J., & Arencibia-Jorge, R. (2016). A review of altmetrics as an emerging discipline for research evaluation. Learned Publishing, 29(4), 229–238. https://doi.org/10.1002/leap.1043.
Görögh, E., Sifacaki, E., Vignoli, M., Gauch, S., Blümel, C., Kraker, P., et al. (2017). Opening up new channels for scholarly review, dissemination, and assessment. In Proceedings of the 13th international symposium on open collaboration—OpenSym’17 (pp. 1–11). New York, New York, USA: ACM Press. https://doi.org/10.1145/3125433.3125452.
Greifeneder, E., Pontis, S., Blandford, A., Attalla, H., Neal, D., & Schlebbe, K. (2018). Researchers’ attitudes towards the use of social networking sites. Journal of Documentation, 74(1), 119–136. https://doi.org/10.1108/JD-04-2017-0051.
Gu, F., & Widén-Wulff, G. (2011). Scholarly communication and possible changes in the context of social media: A Finnish case study. The Electronic Library, 29(6), 762–776. https://doi.org/10.1108/02640471111187999.
Gumpenberger, C., Glänzel, W., & Gorraiz, J. (2016). The ecstasy and the agony of the altmetric score. Scientometrics, 108(2), 977–982. https://doi.org/10.1007/s11192-016-1991-5.
Haustein, S. (2016). Grand challenges in altmetrics: heterogeneity, data quality and dependencies. Scientometrics, 108(1), 413–423. https://doi.org/10.1007/s11192-016-1910-9.
Haustein, S., Bowman, T. D., Holmberg, K., Tsou, A., Sugimoto, C. R., & Larivière, V. (2016). Tweets as impact indicators: Examining the implications of automated “bot” accounts on Twitter. Journal of the Association for Information Science and Technology, 67(1), 232–238. https://doi.org/10.1002/asi.23456.
Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2014). Coverage and adoption of altmetrics sources in the bibliometric community. Scientometrics, 101(2), 1145–1163. https://doi.org/10.1007/s11192-013-1221-3.
Haustein, S., Sugimoto, C., & Larivière, V. (2015). Guest editorial: Social media in scholarly communication. Aslib Journal of Information Management. https://doi.org/10.1108/AJIM-03-2015-0047.
Herman, E., & Nicholas, D. (2019). Scholarly reputation building in the digital age: An activity-specific approach. Review article. El Profesional de La Información, 28(1), 1–31. https://doi.org/10.3145/epi.2019.ene.02.
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429–431. https://doi.org/10.1038/520429a.
Hoffmann, C. P., Lutz, C., & Meckel, M. (2016). A relational altmetric? Network centrality on ResearchGate as an indicator of scientific impact. Journal of the Association for Information Science and Technology, 67(4), 765–775. https://doi.org/10.1002/asi.23423.
Jamali, H. R. (2017). Copyright compliance and infringement in ResearchGate full-text journal articles. Scientometrics, 112(1), 241–254. https://doi.org/10.1007/s11192-017-2291-4.
Jamali, H. R., Nicholas, D., & Herman, E. (2016). Scholarly reputation in the digital age and the role of emerging platforms and mechanisms. Research Evaluation, 25(1), 37–49. https://doi.org/10.1093/reseval/rvv032.
Jeng, W., DesAutels, S., He, D., & Li, L. (2017). Information exchange on an academic social networking site: A multidiscipline comparison on researchgate Q&A. Journal of the Association for Information Science and Technology, 68(3), 638–652. https://doi.org/10.1002/asi.23692.
Jordan, K. (2015). Exploring the ResearchGate score as an academic metric: Reflections and implications for practice. Quantifying and analysing scholarly communication on the web (ASCW’15) (pp. 1–3). Retrieved from http://ascw.know-center.tugraz.at/wp-content/uploads/2015/06/ASCW15jordanresponsekraker-lex.pdf%5CnCopyright.
Kadriu, A. (2013). Discovering value in academic social networks: A case study in ResearchGate. In Proceedings of the ITI 2013 35th international conference on information technology interfaces (pp. 57–62). Zagreb: University Computing Centre—SRCE. https://doi.org/10.2498/iti.2013.0566.
Kjellberg, S., Haider, J., & Sundin, O. (2016). Researchers’ use of social network sites: A scoping review. Library & Information Science Research, 38(3), 224–234. https://doi.org/10.1016/j.lisr.2016.08.008.
Kraker, P., & Lex, E. (2015). A critical look at the ResearchGate score as a measure of scientific reputation. In ASCW’15 workshop at Web Science 2015, (May) (pp. 7–9). https://doi.org/10.5281/zenodo.35401.
Lepori, B., Thelwall, M., & Hoorani, B. H. (2018). Which US and European higher education institutions are visible in ResearchGate and what affects their RG score? Journal of Informetrics, 12(3), 806–818. https://doi.org/10.1016/j.joi.2018.07.001.
Liu, X. Z., & Fang, H. (2018). Which academic papers do researchers tend to feature on researchgate? Information Research, 23(1). http://www.informationr.net/ir/23-1/paper785.html.
Manca, S. (2017). An analysis of ResearchGate and Academia. edu as socio-technical systems for scholars’ networked learning: a multilevel framework proposal. Italian Journal of Educational Technology, 25(3), 20–34. https://doi.org/10.17471/2499-4324/985.
Manca, S. (2018). ResearchGate and Academia.edu as networked socio-technical systems for scholarly communication: A literature review. Research in Learning Technology, 26(1063519), 1–16. https://doi.org/10.25304/rlt.v26.2008.
Martín-Martín, A., Orduna-Malea, E., & Delgado López-Cózar, E. (2018). Author-level metrics in the new academic profile platforms: The online behaviour of the Bibliometrics community. Journal of Informetrics, 12(2), 494–509. https://doi.org/10.1016/j.joi.2018.04.001.
Meier, A., & Tunger, D. (2018). Investigating the transparency and influenceability of altmetrics using the example of the RG score and the ResearchGate platform. Information Services & Use, 38(1–2), 99–110. https://doi.org/10.3233/ISU-180001.
Meishar-Tal, H., & Pieterse, E. (2017). Why do academics use academic social networking sites. The International Review of Research in Open and Distributed Learning, 18(1), 1–22. https://doi.org/10.19173/irrodl.v18i1.2643.
Memisevic, H., Taljic, I., & Hadziomerovic, A. (2017). Making use of H-index: The shape of science at the University of Sarajevo. Acta Informatica Medica, 25(3), 187. https://doi.org/10.5455/aim.2017.25.187-190.
Mingers, J., & Leydesdorff, L. (2015). A review of theory and practice in scientometrics. European Journal of Operational Research, 246(1), 1–19. https://doi.org/10.1016/j.ejor.2015.04.002.
Nentwich, M., & König, R. (2014). Academia goes Facebook? The potential of social network sites in the scholarly realm. In S. Bartling & S. Friesike (Eds.), Opening science (pp. 107–124). Cham: Springer. https://doi.org/10.1007/978-3-319-00026-8_7.
Nicholas, D., Boukacem-Zeghmouri, C., Rodríguez-Bravo, B., Xu, J., Watkinson, A., Abrizah, A., et al. (2017). Where and how early career researchers find scholarly information. Learned Publishing, 30(1), 19–29. https://doi.org/10.1002/leap.1087.
Nicholas, D., Clark, D., & Herman, E. (2016a). ResearchGate: Reputation uncovered. Learned Publishing, 29(3), 173–182. https://doi.org/10.1002/leap.1035.
Nicholas, D., Herman, E., & Clark, D. (2016b). Scholarly reputation building: How does ResearchGate fare? International Journal of Knowledge Content Development & Technology, 6(2), 67–92. https://doi.org/10.5865/IJKCT.2016.6.2.067.
Nicholas, D., Herman, E., Jamali, H. R., Bravo, B. R., Boukacem-Zeghmouri, C., Dobrowolski, T., et al. (2015). New ways of building, showcasing, and measuring scholarly reputation. Learned Publishing, 28(3), 169–183. https://doi.org/10.1087/20150303.
Nicholas, D., Herman, E., Xu, J., Boukacem-Zeghmouri, C., Abdullah, A., Watkinson, A., et al. (2018). Early career researchers’ quest for reputation in the digital age. Journal of Scholarly Publishing, 49(4), 375–396. https://doi.org/10.3138/jsp.49.4.01.
Orduna-Malea, E., Martín-Martín, A., Thelwall, M., & Delgado López-Cózar, E. (2017). Do ResearchGate scores create ghost academic reputations? Scientometrics, 112(1), 443–460. https://doi.org/10.1007/s11192-017-2396-9.
Ortega, J. L. (2015a). Disciplinary differences in the use of academic social networking sites. Online Information Review, 39(4), 520–536. https://doi.org/10.1108/OIR-03-2015-0093.
Ortega, J. L. (2015b). Relationship between altmetric and bibliometric indicators across academic social sites: The case of CSIC’s members. Journal of Informetrics, 9(1), 39–49. https://doi.org/10.1016/j.joi.2014.11.004.
Ortega, J. L. (2016). Social network sites for scientists (1st ed.). Amsterdam: Elsevier.
Ortega, J. L. (2017). Toward a homogenization of academic social sites: A longitudinal study of profiles in Academia.edu, Google Scholar citations and ResearchGate. Online Information Review, 41(6), 812–825. https://doi.org/10.1108/OIR-01-2016-0012.
Procter, R., Williams, R., Stewart, J., Poschen, M., Snee, H., Voss, A., et al. (2010). Adoption and use of Web 2.0 in scholarly communications. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 368(1926), 4039–4056. https://doi.org/10.1098/rsta.2010.0155.
Raffaghelli, J. E., & Manca, S. (2018). Exploring the geographies of academic social network sites from a socio-technical perspective: An investigation of scientific literature in Spanish of scientific literature in Spanish. In M. Bajić, N. Dohn, M. de Laat, P. Jandrić, & T. Ryberg (Eds.), Proceedings of the 11th international conference on networked learning, Zagreb.
Rowlands, I., Nicholas, D., Russell, B., Canty, N., & Watkinson, A. (2011). Social media use in the research workflow. Learned Publishing, 24(3), 183–195. https://doi.org/10.1087/20110306.
Shrivastava, R., & Mahajan, P. (2015). Relationship amongst ResearchGate altmetric indicators and Scopus bibliometric indicators. New Library World, 116(9/10), 564–577. https://doi.org/10.1108/NLW-03-2015-0017.
Singson, M., & Amees, M. (2017). Use of ResearchGate by the research scholars of Pondicherry University: A study. DESIDOC Journal of Library & Information Technology, 37(5), 366. https://doi.org/10.14429/djlit.37.11139.
Sugimoto, C. R., Work, S., Larivière, V., & Haustein, S. (2017). Scholarly use of social media and altmetrics: A review of the literature. Journal of the Association for Information Science and Technology, 68(9), 2037–2062. https://doi.org/10.1002/asi.23833.
Thelwall, M., & Kousha, K. (2015). ResearchGate: Disseminating, communicating, and measuring scholarship? Journal of the Association for Information Science and Technology, 66(5), 876–889. https://doi.org/10.1002/asi.23236.
Thelwall, M., & Kousha, K. (2017). ResearchGate articles: Age, discipline, audience size, and impact. Journal of the Association for Information Science and Technology, 68(2), 468–479. https://doi.org/10.1002/asi.23675.
Torres-Salinas, D., Cabezas-Clavijo, Á., & Jiménez-Contreras, E. (2013). Altmetrics: New indicators for scientific communication in Web 2.0. Comunicar, 21(41), 53–60. https://doi.org/10.3916/C41-2013-05.
Van Noorden, R. (2014). Online collaboration: Scientists and the social network. Nature, 512(7513), 126–129. https://doi.org/10.1038/512126a.
Veletsianos, G., & Kimmons, R. (2012). Networked participatory scholarship: Emergent techno-cultural pressures toward open and digital scholarship in online networks. Computers & Education, 58(2), 766–774. https://doi.org/10.1016/j.compedu.2011.10.001.
Waltman, L. (2016). A review of the literature on citation impact indicators. Journal of Informetrics, 10(2), 365–391. https://doi.org/10.1016/j.joi.2016.02.007.
Wouters, P., & Costas, R. (2012). Users, narcissism and control ? Tracking the impact of scholarly publications in the 21st century. In Proceedings of the 17th international conference on science and technology indicators (pp. 847–857).
Yan, W., & Zhang, Y. (2018). Research universities on the ResearchGate social networking site: An examination of institutional differences, research activity level, and social networks formed. Journal of Informetrics, 12(1), 385–400. https://doi.org/10.1016/j.joi.2017.08.002.
Yan, W., Zhang, Y., & Bromfield, W. (2018). Analyzing the follower–followee ratio to determine user characteristics and institutional participation differences among research universities on ResearchGate. Scientometrics, 115(1), 299–316. https://doi.org/10.1007/s11192-018-2637-6.
Yi, S., Liu, Q., & Yan, W. (2018). How corporations utilize academic social networking website? A case study of Health & Biomedicine Corporations. In H. Chen, Q. Fang, D. Zeng, & J. Wu (Eds.), Lecture notes in computer science (pp. 325–331). Cham: Springer. https://doi.org/10.1007/978-3-030-03649-2_32.
Yu, M.-C., Wu, Y.-C. J., Alhalabi, W., Kao, H.-Y., & Wu, W.-H. (2016). ResearchGate: An effective altmetric indicator for active researchers? Computers in Human Behavior, 55, 1001–1006. https://doi.org/10.1016/j.chb.2015.11.007.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Copiello, S. Research Interest: another undisclosed (and redundant) algorithm by ResearchGate. Scientometrics 120, 351–360 (2019). https://doi.org/10.1007/s11192-019-03124-w
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-019-03124-w