Abstract
Traditional citation analysis methods have been criticized because their theoretical base of statistical counts does not reflect the motive or judgment of citing authors. In particular, self-citations may give undue credits to a cited article or mislead scientific development. This research aims to answer the question of whether self-citation is biased by probing into the motives and context of citations. It takes an integrated and fine-grained view of self-citations by examining them via multiple lenses — polarity, density, and location of citations. In addition, it explores potential moderating effects of citation level and associations among location contexts of citations to the same references for the first time. We analyzed academic publications across different topics and disciplines using both qualitative and quantitative methods. The results provide evidence that self-citations are free of bias in terms of citation density and polarity uncertainty, but they can be biased with respect to positivity and negativity of citations. Furthermore, this study reveals impacts of self-citing behavior on some citation patterns involving citation density, location concentration, and associations. The examination of self-citing behavior from those new perspectives shed new lights on the nature and function of self-citing behavior.
Similar content being viewed by others
References
Athar, A. (2011). Sentiment analysis of citations using sentence structure-based features. Portland, Oregon: Paper presented at the proceedings of the ACL 2011 Student Session.
Bartneck, C., & Kokkelmans, S. (2010). Detecting h-index manipulation through self-citation analysis. Scientometrics, 87(1), 85–98.
Blau, J. R. (2000). Group enmity and accord: The commercial Press in Three American Cities. Social Science History, 24(2), 395–413.
Bonzi, S., & Snyder, H. W. (1991). Motivations for citation: A comparison of self citation and citation to others. Scientometrics, 21(2), 245–254.
Bowers, J. S. (2009). On the biological plausibility of grandmother cells: Implications for neural network theories in psychology and neuroscience. Psychological Review, 116(1), 220–251.
Bowers, J. S. (2010). More on grandmother cells and the biological implausibility of PDP models of cognition: A reply to Plaut and McClelland (2010) and Quian Quiroga and Kreiman (2010). Psychological Review, 117(1), 300–306 discussion 289–390, 297–309, 306–308.
Brooks, T. A. (1985). Private acts and public objects: An investigation of citer motivations. Journal of the American Society for Information Science, 36(4), 223–229.
Brooks, T. A. (1986). Evidence of complex citer motivations. Journal of the American Society for Information Science, 37(1), 34–36.
Brown, R. J. C. (2009). A simple method for excluding self-citation from the h-index: The b-index. Online Information Review, 33(6), 1129–1136.
Butler, L., & Visser, M. S. (2006). Extending citation analysis to non-source items. Scientometrics, 66(2), 327–343.
Cano, V. (1989). Citation behavior: Classification, utility, and location. Journal of the American Society for Information Science, 40(4), 284–290.
Case, D. O., & Higgins, G. M. (2000). How can we investigate citation behavior? A study of reasons for citing literature in communication. Journal of the American Society for Information Science, 51(7), 635–645.
Catalinia, C., Laceterab, N., & Oettlc, A. (2006). The incidence and role of negative citations in science. Proceedings of the National Academy of Sciences of the United States of America (PNAS), 112(45), 13823–13826.
Chubin, D. E., & Moitra, S. D. (1975). Content analysis of references: Adjunct or alternative to citation counting? Social Studies of Science, 5(4), 423–441.
Cole, J., & Cole, S. (1971). Measuring the quality of sociological research: Problems in the use of the science citation index. The American Sociologist, 6, 23–29.
Couto, F. M., Pesquita, C., Grego, T., & Veríssimo, P. (2009). Handling self-citations using Google scholar. Cybermetrics: International Journal of Scientometrics, Informetrics and Bibliometrics, 13(1).
Diodato, V. (1994). Dictionary of bibliometrics. New York: Haworth Press.
Dwivedi, Y., Lal, B., Mustafee, B., & Williams, M. (2009). Profiling a decade of information systems frontiers’ research. Information Systems Frontiers, 11(1), 87–102.
Elo, S., & Kyngas, H. (2008). The qualitative content analysis process. Journal of Advanced Nursing, 62(1), 107–115.
Ferligoj, A., Maričić, S., Pifat, G., & Spaventi, J. (1988). Cluster analysis of citation histories from an institutional setting. In N. Tudor-Šilović & I. Mihel (Eds.), Information research: Research methods in library and information science. London: Taylor Graham.
Ferrara, E., & Romero, A. E. (2013). Scientific impact evaluation and the effect of self-citations: Mitigating the bias by discounting the h-index. Journal of the American Society for Information Science and Technology, 64(11), 2332–2339.
Fooladi, M., Salehi, H., Yunus, M. M., Farhadi, M., Chadegani, A. A., Farhadi, H., & Ebrahim, N. A. (2013). Does criticisms overcome the praises of journal impact factor? Asian Social Science, 9(5).
Fowler, J. H., & Aksnes, D. W. (2007). Does self-citation pay? Scientometrics, 72(3), 427–437.
Garfield, E. (1972). Citation analysis as a tool in journal evaluation: Journals can be ranked by frequency and impact of citations for science policy studies. Science, 178(4060), 471–479.
Glänzel, W., Debackere, K., Thijs, B., & Schubert, A. (2006). A concise review on the role of author self-citations in information science, bibliometrics and science policy. Scientometrics, 67(2), 263–277.
Harzing, A.-W., & van der Wal, R. (2009). A Google scholar h-index for journals: An alternative metric to measure journal impact in economics and business. Journal of the American Society for Information Science and Technology, 60(1), 41–46.
Hirsch, J. E. (2005). An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.
Hirsch, J. E. (2010). An index to quantify an individual’s scientific research output that takes into account the effect of multiple coauthorship. Scientometrics, 85(3), 741–754.
Hodge, D. R., & Lacasse, J. R. (2011). Evaluating journal quality: Is the H-index a better measure than impact factors? Research on Social Work Practice, 21(2), 222–230.
Hyland, K. (2003). Self-citation and self-reference: Credibility and promotion in academic publication. Journal of the American Society for Information Science and Technology, 54(3), 251–259.
Ioannidis, J. P. A. (2015). A generalized view of self-citation: Direct, co-author, collaborative, and coercive induced self-citation. Journal of Psychosomatic Research, 78(1), 7–11.
Kapoor, K. K., Tamilmani, K., Rana, N. P., Patil, P., Dwivedi, Y. K., & Nerur, S. (2018). Advances in social media research: Past, present and future. Information Systems Frontiers, 20(3), 531–558.
Kulkarni, Aziz, Shams, & Busse (2011). Author self-citation in the general medicine literature. PLoS One, 6(6), e20885.
Labkovsky, E., & Rosenfeld, J. P. (2012). The P300-based, complex trial protocol for concealed information detection resists any number of sequential countermeasures against up to five irrelevant stimuli. Applied Psychophysiology and Biofeedback, 37(1), 1–10.
Lehky, S. R., Kiani, R., Esteky, H., & Tanaka, K. (2011). Statistics of visual responses in primate inferotemporal cortex to object stimuli. Journal of Neurophysiology, 106(3), 1097–1117.
Lowry, P. B., Moody, G. D., Gaskin, J., Galletta, D. F., Humpherys, S. L., Barlow, J. B., & Wilson, D. W. (2013). Evaluating journal quality and the association for information systems senior scholars' journal basket via bibliometric measures: Do expert journal assessments add value? MIS Quarterly., 37(4), 993–1012.
Lutz, B., & Hans-Dieter, D. (2008). What do citation counts measure? A review of studies on citing behavior. Journal of Documentation, 64(1), 45–80.
Maričić, S., Spaventi, J., Pavičić, L., & Pifat-Mrzljak, G. (1998). Citation context versus the frequency counts of citation histories. Journal of the American Society for Information Science, 49(6), 530–540.
Mayring, P. (2000). Qualitative content analysis. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 1(2).
McCain, K., & Turner, K. (1989). Citation context analysis and aging patterns of journal articles in molecular genetics. Scientometrics, 17(1–2), 127–163.
Mertens, R., & Allen, J. J. B. (2008). The role of psychophysiology in forensic assessments: Deception detection, ERPs, and virtual reality mock crime scenarios. Psychophysiology, 45(2), 286–298. https://doi.org/10.1111/j.1469-8986.2007.00615.x.
Moravcsik, M. J., & Murugesan, P. (1975). Some results on the function and quality of citations. Social Studies of Science, 5(1), 86–92.
Nicolaisen, J. (2007). Citation analysis. Annual Review of Information Science and Technology, 41(1), 609–641.
O'Leary, D. E. (2008). The relationship between citations and number of downloads in decision support systems. Decision Support Systems, 45(4), 972–980.
Osareh, F. (1996). Bibliometrics, Citation Analysis and Co-Citation Analysis (Vol. 46, p. 149). A Review of Literature I Libri.
Peritz, B. (1983). A classification of citation roles for the social sciences and related fields. Scientometrics, 5(5), 303–312.
Plaut, D. C., & McClelland, J. L. (2010). Locating object knowledge in the brain: Comment on Bowers's (2009) attempt to revive the grandmother cell hypothesis. Psychological Review, 117(1), 284–288.
Quiroga, R., & Kreiman, G. (2010). Measuring sparseness in the brain: Comment on bowers (2009). Psychological Review, 117, 291–297.
Rad, A. E., Shahgholi, L., & Kallmes, D. (2012). Impact of self-citation on the H index in the field of academic radiology. Academic Radiology, 19(4), 455–457.
Rodríguez-Ruiz, Ó. (2009). The citation indexes and the quantification of knowledge. Journal of Educational Administration, 47(2), 250–266.
Rosenfeld, J. P., Soskins, M., Bosh, G., & Ryan, A. (2004). Simple, effective countermeasures to P300-based tests of detection of concealed information. Psychophysiology, 41(2), 205–219. https://doi.org/10.1111/j.1469-8986.2004.00158.x.
Sammarco, P. W. (2008). Journal visibility, self-citation, and reference limits: Influences on Impact Factor and author performance review. Ethics in Science and Environmental Politics, 8, 121–125.
Sanderson, M. (2008). Revisiting h measured on UK LIS and IR academics. Journal of the American Society for Information Science and Technology, 59(7), 1184–1190.
Shadish, W. R., Tolliver, D., Gray, M., & Sen Gupta, S. K. (1995). Author judgements about works they cite: Three studies from psychology journals. Social Studies of Science, 25(3), 477–498.
Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27(3), 379–423.
Small, H. G. (1978). Cited documents as concept symbols. Social Studies of Science, 8(3), 327–340.
Snyder, H., & Bonzi, S. (1998). Patterns of self-citation across disciplines (1980-1989). Journal of Information Science, 24(6), 431–435.
Straub, D. W., & Anderson, C. (2009). Journal self - citation V I: Forced journal self - citation – Common, appropriate, ethical? Communications of the Association for Information Systems, 25, 57–66.
Swales, J. (1986). Citation analysis and discourse analysis. Applied Linguistics, 7(1), 39–56.
Tang, E. K., Suganthan, P. N., & Yao, X. (2006). An analysis of diversity measures. Machine Learning, 65(1), 247–271.
Thombs, B. D., Levis, A. W., Razykov, I., Syamchandra, A., Leentjens, A. F. G., Levenson, J. L., & Lumley, M. A. (2015). Potentially coercive self-citation by peer reviewers: A cross-sectional study. Journal of Psychosomatic Research, 78(1), 1–6.
Waltman, L., & Eck, N. J. v. (2012). The inconsistency of the h-index. Journal of the American Society for Information Science and Technology, 63(2), 406–415. https://doi.org/10.1002/asi.21678.
Wang, N., Liang, H., Jia, Y., Ge, S., Xue, Y., & Wang, Z. (2016). Cloud computing research in the IS discipline: A citation/co-citation analysis. Decision Support Systems, 86, 35–47.
White, H. D. (2004). Citation analysis and discourse analysis revisited. Applied Linguistics, 25(1), 89–116.
Whitehouse, G. H. (2001). Citation rates and impact factors: Should they matter? The British Journal of Radiology, 74(877), 1–3.
Zhang, C.-T. (2013). The h’-index, effectively improving the h-index based on the citation distribution. PLoS One, 8(4), e59912.
Zhang, G., Ding, Y., & Milojević, S. (2013). Citation content analysis (CCA): A framework for syntactic and semantic analysis of citation content. Journal of the American Society for Information Science and Technology, 64(7), 1490–1503.
Zhou, L., Zhang, D., & Sung, Y.-w. (2013). The effects of group factors on deception detection performance. Small Group Research, 44(3), 272–297.
Acknowledgements
The authors would like to thank the researchers and students who provided assistance with encoding the content and context of paper citations. This research is supported in part by the National Science Foundation (SES-152768, CNS-1704800). Any opinions, findings or recommendations expressed here are those of the authors and are not necessarily those of the sponsors of this research.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhou, L., Amadi, U. & Zhang, D. Is Self-Citation Biased? An Investigation via the Lens of Citation Polarity, Density, and Location. Inf Syst Front 22, 77–90 (2020). https://doi.org/10.1007/s10796-018-9889-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10796-018-9889-9