Skip to main content
SpringerLink
Log in

Can social media usage of scientific literature predict journal indices of AJG, SNIP and JCR? An altmetric study of economics

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Altmetrics are often praised as an alternative or complement to classic bibliometric metrics, especially in the social sciences discipline. However, empirical investigations of altmetrics concerning the social sciences are scarce. This study investigates the extent to which economic research is shared on social media platforms with an emphasis on mentions in policy documents in addition to other mentions such as Twitter or Facebook. Moreover, this study explores machine learning models to predict the likelihood of a research article being classified into the top-quality tier of a journal ranking based on the altmetric mentions. The included journal rankings are the academic journal guide (AJG), source normalized impact per paper (SNIP) and journal citation reports (JCR). The investigated journals have been selected based on the AJG list and extracted from Altmetric.com data. After applying extensive data cleaning on the extracted data, a final set of 55,560 journal article records is obtained. The results indicate that the average number of policy mentions of the publications of economics journals is higher than the other subject areas included in the AJG list. Moreover, the publications in top-ranking economic journals are more likely to have a higher average number of policy mentions. Policy and Twitter mentions are presented as the most significant and informative social media mentions in demonstrating the broader impact and dissemination of Economics discipline followed by Blogs, Facebook, Wikipedia, and News. The results show that Support Vector Machine and Logistic Regression performed best in classifying the journal ranking tiers i.e. SNIP-based with 77% accuracy, JCR-based with 71% accuracy, and AJG-based with 66% accuracy. The models classified the ranking tier AJG18 with lower accuracy than SNIP and JCR. This might be because the AJG18 rankings are based on expert opinion, whereas SNIP and JCR are based on citations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Aung, H. H., Zheng, H., Erdt, M., Aw, A. S., Sin, S. C. J., & Theng, Y. L. (2019). Investigating familiarity and usage of traditional metrics and counts. Journal of the Association for Information Science and Technology, 70(8), 872–887.

    Article  Google Scholar 

  • Bailey, C., Kale, B., Walker, J., Siravuri, H. V., Alhoori, H., & Papka, M. E. (2017). Exploring features for predicting policy citations. In 2017 ACM/IEEE Joint Conference on Digital Libraries (JCDL) (pp. 1–2). IEEE.

  • Bornmann, L., Haunschild, R., & Marx, W. (2016). Policy documents as sources for measuring societal impact: How often is climate change research mentioned in policy-related documents? Scientometrics, 109, 1477–1495.

    Article  Google Scholar 

  • CABS. (2018). AJG academic journal guide—methodology. Retrieved from Chartered Association of Business Schools. Retrieved from https://charteredabs.org/academic-journal-guide-2018/.

  • Costas, R., Zahedi, Z., & Wouters, P. (2015). Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Science and Technology, 66(10), 2003–2019.

    Article  Google Scholar 

  • Drongstrup, D., Malik, S., & Hassan, S.U. (2019). Altmetrics study of economics. In Paper Presented at the 17th International Conference on Scientometrics and Informetrics, Rome, Italy.

  • Eysenbach, G. (2011). Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. Journal of Medical Internet Research, 13(4), e123. https://doi.org/10.2196/jmir.2012.

    Article  Google Scholar 

  • Hammarfelt, B. (2014). Using altmetrics for assessing research impact in the humanities. Scientometrics, 101(2), 1419–1430.

    Article  Google Scholar 

  • Hassan, S. U., Aljohani, N. R., Idrees, N., Sarwar, R., Nawaz, R., Martínez-Cámara, E., et al. (2020a). Predicting literature’s early impact with sentiment analysis in Twitter. Knowledge-Based Systems, 192, 105383. https://doi.org/10.1016/j.knosys.2019.105383.

    Article  Google Scholar 

  • Hassan, S. U., Aljohani, N. R., Shabbir, M., Ali, U., Iqbal, S., Sarwar, R., et al. (2020b). Tweet coupling: A social media methodology for clustering scientific publications. Scientometrics. https://doi.org/10.1007/s11192-020-03499-1.

    Article  Google Scholar 

  • Hassan, S. U., Bowman, T. D., Shabbir, M., Akhtar, A., Imran, M., & Aljohani, N. R. (2019). Influential tweeters in relation to highly cited articles in altmetric big data. Scientometrics, 119(1), 481–493.

    Article  Google Scholar 

  • Hassan, S.-U., Imran, M., Gillani, U., Aljohani, N. R., Bowman, T. D., & Didegah, F. (2017). Measuring social media activity of scientific literature: An exhaustive comparison of scopus and novel altmetrics big data. Scientometrics, 113(2), 1037–1057.

    Article  Google Scholar 

  • Hassan, S. U., Imran, M., Iqbal, S., Aljohani, N. R., & Nawaz, R. (2018). Deep context of citations using machine-learning models in scholarly full-text articles. Scientometrics, 117(3), 1645–1662.

    Article  Google Scholar 

  • Hassan, S. U., Iqbal, S., Aljohani, N. R., Alelyani, S., & Zuccala, A. (2020c). Introducing the ‘alt-index’for measuring the social visibility of scientific research. Scientometrics, 123(3), 1407–1419.

    Article  Google Scholar 

  • Haunschild, R., & Bornmann, L. (2017). How many scientific papers are mentioned in policy-related documents? An empirical investigation using web of science and altmetric data. Scientometrics, 110(3), 1209–1216.

    Article  Google Scholar 

  • Haustein, S., Peters, I., Sugimoto, C. R., Thelwall, M., & Larivière, V. (2014). Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature. Journal of the Association for Information Science and Technology, 65(4), 656–669.

    Article  Google Scholar 

  • Holmberg, K., & Thelwall, M. (2014). Disciplinary differences in twitter scholarly communication. Scientometrics, 101(2), 1027–1042.

    Article  Google Scholar 

  • Jacso, P. (2009). Five-year impact factor data in the journal citation reports, online information review (pp. 603–614). Bingley: Emerald Group Publishing Limited.

    Google Scholar 

  • Kelly, E. J. (2017). Altmetrics and archives. Journal of Contemporary Archival Studies, 4(1), 1.

    Google Scholar 

  • Lewis, D. D. (1998). Naive (Bayes) at forty: The independence assumption in information retrieval. In European Conference on Machine Learning (pp. 4–15). Springer, Berlin, Heidelberg.

  • Liu, J. (2014). New source alert: Policy documents. Retrieved from https://www.Altmetric.com/blog/new-source-alert-policy-documents/Moed.

  • Moed, F. (2010). Measuring contextual citation impact of scientific journals. Journal of Informetrics, 4(3), 265–277.

    Article  Google Scholar 

  • Nederhof, A. J. (2006). Bibliometric monitoring of research performance in the social sciences and the humanities: A review. Scientometrics, 66(1), 81–100.

    Article  MathSciNet  Google Scholar 

  • Nuredini, K. & Peters, I. (2015). Economic and business studies journals and readership information from Mendeley. Re: Inventing Information Science in the Networked Society. In Proceedings of the 14th International Symposium on Information Science, Zadar/Croatia, (ISI 2015) (pp. 380–392).

  • Nuredini, K. & Peters, I. (2016). Enriching the knowledge of altmetrics studies by exploring social media metrics for economic and business studies journals. In Proceedings of the 21st International Conference on Science and Technology Indicators (STI Conference 2016).

  • Said, A., Bowman, T. D., Abbasi, R. A., Aljohani, N. R., Hassan, S. U., & Nawaz, R. (2019). Mining network-level properties of Twitter altmetrics data. Scientometrics, 120(1), 217–235.

    Article  Google Scholar 

  • Segal, M. R. (2004). Machine learning benchmarks and random forest regression. Netherlands: Kluwer Academic Publishers.

    Google Scholar 

  • Shuai, X., Pepe, A., & Bollen, J. (2012). How the scientific community reacts to newly submitted preprints: Article downloads, Twitter mentions, and citations. PLoS ONE, 7(11), e47523. https://doi.org/10.1371/journal.pone.0047523.

    Article  Google Scholar 

  • Suykens, J. A., & Vandewalle, J. (1999). Least squares support vector machine classifiers. Neural Processing Letters, 9(3), 293–300.

    Article  Google Scholar 

  • Tattersall, A., & Carroll, C. (2018). What can altmetric. com tell us about policy citations of research? An analysis of altmetric. com data for research articles from the University of sheffield. Frontiers in Research Metrics and Analytics. https://doi.org/10.3389/frma.2017.00009.

    Article  Google Scholar 

  • Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R. (2013). Do altmetrics work? Twitter and ten other social web services. PLoS ONE, 8(5), e64841.

    Article  Google Scholar 

  • Waltman, L., van Eck, N. J., van Leeuwen, T. N., & Visser, M. S. (2013). Some modifications to the snip journal impact indicator. Journal of Informetrics, 7(2), 272–285.

    Article  Google Scholar 

  • Wright, R. E. (1995). Logistic regression. Washington: American Psychological Association.

    Google Scholar 

  • Zahedi, Z., Fenner, M., & Costas, R. (2014). How consistent are altmetrics providers? Study of 1000 PLOS ONE publications using the PLOS ALM, Mendeley and altmetric. com APIs. In Altmetrics 14. Workshop at the Web Science Conference, Bloomington, USA.

Download references

Acknowledgements

This article is an extended version of research in progress presented at the 17th International Conference on Scientometrics and Informetrics, Rome (Italy), 2–5 September 2019 (Drongstrup et al. 2019). The authors (Saeed-Ul Hassan and Salem Alelyani) are grateful for the financial support received from King Khalid University for this research under Grant No. 239, 2019.

Author information

Authors and Affiliations

  1. Research and Analysis Section, University Library of Southern Denmark, Campusvej 55, 5230, Odense, Denmark

    Dorte Drongstrup

  2. Information Technology University, 346-B, Ferozepur Road, Lahore, Pakistan

    Shafaq Malik, Iqra Safder & Saeed-Ul Hassan

  3. Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Kingdom of Saudi Arabia

    Naif Radi Aljohani

  4. Center for Artificial Intelligence (CAI), King Khalid University, P.O. Box 9004, Abha, 61413, Saudi Arabia

    Salem Alelyani

  5. College of Computer Science, King Khalid University, P.O. Box 9004, Abha, 61413, Saudi Arabia

    Salem Alelyani

Authors
  1. Dorte Drongstrup
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shafaq Malik
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Naif Radi Aljohani
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Salem Alelyani
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Iqra Safder
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Saeed-Ul Hassan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Saeed-Ul Hassan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Drongstrup, D., Malik, S., Aljohani, N.R. et al. Can social media usage of scientific literature predict journal indices of AJG, SNIP and JCR? An altmetric study of economics. Scientometrics 125, 1541–1558 (2020). https://doi.org/10.1007/s11192-020-03613-3

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-020-03613-3

Keywords

Search

Navigation