Abstract
To explore whether there are other factors than count and sentiment that should be incorporated in evaluating research papers with social media mentions, this paper analyses the content of tweets linking to the top 100 papers of 2015 taken from www.altmetric.com, focusing on the goals, functions and features of research. We discuss three basic issues inherent in using tweets for research evaluation: whose tweets can be used to assess a paper, what objects can be evaluated, and how to score the paper according to each tweet. We suggest that tweets written by those involved in publication of the paper in question should not be included in the paper’s evaluation. Tweets unrelated to the content of the paper should also be excluded. Because controversies in research are inevitable and difficult to resolve, we suggest omitting somewhat supportive and negative tweets in research evaluation. Logically, neutral tweets (such as those linking to, and excerpts from, papers) express a degree of compliment, agreement, interest, or surprise, albeit less so than the tweets explicitly expressing these sentiments. Recommendation tweets also reflect one or more of these sentiments. Expansion tweets, which are inspired by the papers, reflect the function of research. Therefore, we suggest giving a higher weight to praise, agreement, interest, surprise, recommendation and expansion tweets linking to an academic paper than neutral tweets when scoring a paper. Issues related to electronic publishing and social media as learned from tweets are also discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
This may be a typo, as we know the frequency of the wave used in a microwave oven is 2.4 GHz.
References
Barber, B. (1961). Resistance by scientists to scientific discovery—This source of resistance has yet to be given scrutiny accorded religious and ideological sources. Science, 134(347), 596–602.
Bird, A. (2010). The epistemology of science—A bird’s-eye view. Synthese, 175(1), 5–16.
Bornmann, L. (2014a). Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics. Journal of Informetrics, 8(4), 895–903.
Bornmann, L. (2014b). Validity of altmetrics data for measuring societal impact: A study using data from Altmetric and F1000Prime. Journal of Informetrics, 8(4), 935–950.
Bornmann, L. (2015). Alternative metrics in scientometrics: A meta-analysis of research into three altmetrics. Scientometrics, 103(3), 1123–1144.
Bornmann, L. (2016). Letter to the editor what do altmetrics counts mean? A plea for content analyses. Journal of the Association for Information Science and Technology, 67(4), 1016–1017.
Bornmann, L., & Daniel, H.-D. (2008). What do citation counts measure? A review of studies on citing behaviour. Journal of Documentation, 64(1), 45–80.
Bornmann, L., & Haunschild, R. (2016). How to normalize Twitter counts? A first attempt based on journals in the Twitter Index. Scientometrics, 107(3), 1405–1422.
Chen, J. Y. (2015). Virtue and the scientist: Using virtue ethics to examine science’s ethical and moral challenges. Science and Engineering Ethics, 21(1), 75–94.
Costas, R., van Leeuwen, T. N., & Bordons, M. (2010). Self-citations at the meso and individual levels: Effects of different calculation methods. Scientometrics, 82(3), 517–537.
Dallmeier-Tiessen, S., Darby, R., Goerner, B., Hyppoelae, J., Igo-Kemenes, P., Kahn, D., et al. (2011). Highlights from the SOAP project survey. What scientists think about open access publishing. arXiv 1101.5260.
de Winter, J. C. F. (2015). The relationship between tweets, citations, and article views for PLOS ONE articles. Scientometrics, 102(2), 1773–1779.
Fang, H. (2015). A discussion on governmental research grants. Science and Engineering Ethics, 21(5), 1285–1296.
Haustein, S. (2014). Readership metrics. In B. Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics: Harnessing multi-dimensional indicators of performance (pp. 327–344). Cambridge, MA: MIT Press.
Haustein, S., Costas, R., & Larivière, V. (2015). Characterizing social media metrics of scholarly papers: The effect of document properties and collaboration patterns. PLoS ONE, 10(3), e0120495.
Haustein, S., Larivière, V., Thelwall, M., Amyot, D., & Peters, I. (2014a). Tweets vs. Mendeley readers: How do these two social media metrics differ? IT Information Technology, 56(5), 207–215.
Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2014b). Coverage and adoption of altmetrics sources in the bibliometric community. Scientometrics, 101(2), 1145–1163.
Holmberg, K., Bowman, T. D., Haustein, S., & Peters, I. (2014). Astrophysicists’ conversational connections on Twitter. PLoS ONE, 9(8), e106086.
Holmberg, K., & Thelwall, M. (2014). Disciplinary differences in Twitter scholarly communication. Scientometrics, 101(2), 1027–1042.
Jose, D. (2007). Falsificationism and the structure of theories: The Popper–Kuhn controversy about the rationality of normal science. Studies in History and Philosophy of Science, 38(3), 543–554.
Karle, J. (1997). The role of motivation in scientific research. 4. The nature of basic research and its implications. Interdisciplinary Science Reviews, 22(1), 78–84.
Krishnamurthy, B., Gill, P., & Arlitt, M. (2008). A few chirps about Twitter. In Proceedings of the first workshop on Online social networks (pp. 19–24). New York: ACM.
Laakso, M., & Björk, B. C. (2012). Anatomy of open access publishing: A study of longitudinal development and internal structure. BMC Medicine, 10(1), 124.
Laakso, M., & Björk, B. C. (2013). Delayed open access: An overlooked high-impact category of openly available scientific literature. Journal of the American Society for Information Science and Technology, 64(7), 1323–1329.
Laakso, M., Welling, P., Bukvova, H., Nyman, L., Björk, B. C., & Hedlund, T. (2011). The development of open access journal publishing from 1993 to 2009. PLoS ONE, 6(6), e20961.
Mizrahi, M. (2013). What is scientific progress? Lessons from scientific practice. Journal for General Philosophy of Science, 44(2), 375–390.
Mohammadi, E., Thelwall, M., Haustein, S., & Larivière, V. (2015). Who reads research articles? An altmetrics analysis of Mendeley user categories. Journal of the Association for Information Science and Technology, 66(9), 1832–1846.
Mutanen, A. (2016). Pedagogy as a framework for a proper dialogue between science and literature. Philosophia, 44(1), 167–180.
Osterrieder, A. (2013). The value and use of social media as communication tool in the plant sciences. Plant Methods, 9(1), 26.
Philip, A. E. B. (2012). Anticipatory ethics for emerging technologies. Nanoethics, 6(1), 1–13.
Popper, K. R. (2002). The logic of scientific discovery. London: Routledge.
Rousseau, R., & Ye, F. Y. (2013). A multi-metric approach for research evaluation. Chinese Science Bulletin, 58(26), 3288–3290.
Schreiber, M. (2008). The influence of self-citation corrections on Egghe’s g index. Scientometrics, 76(1), 187–200.
Tahamtan, I., Afshar, A. S., & Ahamdzadeh, K. (2016). Factors affecting number of citations: A comprehensive review of the literature. Scientometrics, 107(3), 1195–1225.
Thelwall, M. (2014). Heart and soul: Sentiment strength detection in the social web with sentistrength. Retrieved January 16, 2016 from http://sentistrength.wlv.ac.uk/documentation/SentiStrengthChapter.pdf.
Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R. (2013a). Do altmetrics work? Twitter and ten other social web services. PLoS ONE, 8(5), e64841.
Thelwall, M., Tsou, A., Weingart, S., Holmberg, K., & Haustein, S. (2013b). Tweeting links to academic articles. Cybermetrics, 17(1), 1–8.
Wang, L. L., Liu, X. Z., & Fang, H. (2015). Investigation of the degree to which articles supported by research grants are published in open access health and life sciences journals. Scientometrics, 104(2), 511–528.
Acknowledgements
The authors thank the anonymous reviewers for useful suggestions for improving this contribution.
Funding
Humanities and Social Sciences Foundation of the Ministry of Education of China (16YJE870002).
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
See Table 3.
Rights and permissions
About this article
Cite this article
Liu, X.Z., Fang, H. What we can learn from tweets linking to research papers. Scientometrics 111, 349–369 (2017). https://doi.org/10.1007/s11192-017-2279-0
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-017-2279-0