Abstract
This paper aims to determine the factors significantly predicting the future citation rates of conference papers. Whereas a large body of bibliometric studies has investigated the multiple factors predicting future citation rates, the attention has been paid mainly on journal articles. This study analyzes 43,463 papers from 81 conference series in the ‘Information Science’ and ‘Computer Science’ fields and examines the contributions of conference-related factors to the citation rates of the conference papers. More specifically, this paper assesses the following conference related factors as being potentially predictive factors of citation rates: longevity and names of the conference series, the number of presented papers at individual conferences, acceptance rates, the seasons of conferences, the content similarity of the presented papers at a conference, the degree of the authors’ international collaborations and the records of the best paper awards at conferences. The regression results illustrate that all of the factors were significantly predictive to the future citations of the conference papers. The factors that contributed the most to explain the citations of the conference papers include: the degree of the authors’ international collaborations at individual conferences, the records of best paper awards and the acceptance rates of individual conferences.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Pennsilvania State University; Syracuse University; Temple University; University of California—Irvnine; University of Florida; University of Illinois—Urbana—Champaign; University of Indiana; University of Maryland—College Park; University of North Carolina; University of Pennsilvania; University of Pittsburgh; University of Southern California; University of Washington (in alphabetic order).
In the line of work by Wainer et al. (2011), which substantiated the citation rate changes in the same discipline with this study, the citation number ‘10’ was chosen as the threshold to determine if a given paper is highly cited or not.
References
Barbosa, S. D. J., Silveira, M. S., & Gasparini, I. (2017). What publications metadata tell us about the evolution of a scientific community: The case of the Brazilian human–computer interaction conference series. Scientometrics, 110(1), 275–300. https://doi.org/10.1007/s11192-016-2162-4.
Bartneck, C., & Hu, J. (2009). Scientometric analysis of the CHI proceedings. Paper presented at the proceedings of the SIGCHI conference on human factors in computing systems, Boston, MA, USA.
Borgman, C. L., & Furner, J. (2002). Scholarly communication and bibliometrics. Annual Review of Information Science and Technology, 36(1), 2–72. https://doi.org/10.1002/aris.1440360102.
Bornmann, L., Marx, W., Gasparyan, A. Y., & Kitas, G. D. (2012). Diversity, value and limitations of the journal impact factor and alternative metrics. Rheumatology International, 32(7), 1861–1867.
Butler, L., & Visser, M. S. (2006). Extending citation analysis to non-source items. Scientometrics, 66(2), 327–343. https://doi.org/10.1007/s11192-006-0024-1.
Chan, H. C., Hee-Woong, K., & Chee, T. W. (2006). Information systems citation patterns from International Conference on Information Systems articles. Journal of the American Society for Information Science and Technology, 57(9), 1263–1274. https://doi.org/10.1002/asi.20413.
Chen, J., & Konstan, J. A. (2010). Conference paper selectivity and impact. Communications of the ACM, 53(6), 79–83. https://doi.org/10.1145/1743546.1743569.
Eckmann, M., Rocha, A., & Wainer, J. (2012). Relationship between high-quality journals and conferences in computer vision. Scientometrics, 90(2), 617–630. https://doi.org/10.1007/s11192-011-0527-2.
Elshawi, R., & Sakr, S. (2016). On analyzing the impact of authors and their collaboration patterns in the major computer algorithms research conferences. COLLNET Journal of Scientometrics and Information Management, 10(1), 155–173. https://doi.org/10.1080/09737766.2016.1177951.
Franceschet, M. (2010). A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar. Scientometrics, 83(1), 243–258. https://doi.org/10.1007/s11192-009-0021-2.
Freyne, J., Coyle, L., Smyth, B., & Cunningham, P. (2010). Relative status of journal and conference publications in computer science. Communications of the ACM, 53(11), 124–132. https://doi.org/10.1145/1839676.1839701.
Gargouri, Y., Hajjem, C., Larivière, V., Gingras, Y., Carr, L., Brody, T., et al. (2010). Self-selected or mandated, open access increases citation impact for higher quality research. PLoS ONE, 5(10), e13636. https://doi.org/10.1371/journal.pone.0013636.
Glänzel, W., Schlemmer, B., Schubert, A., & Thijs, B. (2006). Proceedings literature as additional data source for bibliometric analysis. Scientometrics, 68(3), 457–473. https://doi.org/10.1007/s11192-006-0124-y.
Han, P., Shi, J., Li, X., Wang, D., Shen, S., & Su, X. (2014). International collaboration in LIS: Global trends and networks at the country and institution level. Scientometrics, 98(1), 53–72. https://doi.org/10.1007/s11192-013-1146-x.
Harzing, A.-W., & Alakangas, S. (2016). Google Scholar, Scopus and the Web of Science: A longitudinal and cross-disciplinary comparison. Scientometrics, 106(2), 787–804. https://doi.org/10.1007/s11192-015-1798-9.
Hoekman, J., Frenken, K., & Tijssen, R. J. W. (2010). Research collaboration at a distance: Changing spatial patterns of scientific collaboration within Europe. Research Policy, 39(5), 662–673. https://doi.org/10.1016/j.respol.2010.01.012.
Ibáñez, A., Bielza, C., & Larrañaga, P. (2013). Relationship among research collaboration, number of documents and number of citations: a case study in Spanish computer science production in 2000–2009. Scientometrics, 95(2), 689–716. https://doi.org/10.1007/s11192-012-0883-6.
Keith, T. Z. (2014). Multiple regression and beyond: An introduction to multiple regression and structural equation modeling. London: Routledge.
Kim, M. C., & Chen, C. (2015). A scientometric review of emerging trends and new developments in recommendation systems. Scientometrics, 104(1), 239–263. https://doi.org/10.1007/s11192-015-1595-5.
Laender, A. H. F., de Lucena, C. J. P., Maldonado, J. C., de Souza e Silva, E., & Ziviani, N. (2008). Assessing the research and education quality of the top Brazilian Computer Science graduate programs. SIGCSE Bulletin, 40(2), 135–145. https://doi.org/10.1145/1383602.1383654.
Larivière, V., Sugimoto, C. R., & Cronin, B. (2012). A bibliometric chronicling of library and information science’s first hundred years. Journal of the American Society for Information Science and Technology, 63(5), 997–1016. https://doi.org/10.1002/asi.22645.
Lee, D. H., & Brusilovsky, P. (2017). Improving personalized recommendations using community membership information. Information Processing and Management, 53(5), 1201–1214. https://doi.org/10.1016/j.ipm.2017.05.005.
Li, X., Rong, W., Shi, H., Tang, J., & Xiong, Z. (2018). The impact of conference ranking systems in computer science: A comparative regression analysis. Scientometrics, 116(2), 879–907. https://doi.org/10.1007/s11192-018-2763-1.
Lisée, C., Larivière, V., & Archambault, É. (2008). Conference proceedings as a source of scientific information: A bibliometric analysis. Journal of the American Society for Information Science and Technology, 59(11), 1776–1784. https://doi.org/10.1002/asi.20888.
Loizides, O.-S., & Koutsakis, P. (2017). On evaluating the quality of a computer science/computer engineering conference. Journal of Informetrics, 11(2), 541–552. https://doi.org/10.1016/j.joi.2017.03.008.
Martins, W. S., Gonçalves, M. A., Laender, A. H. F., & Ziviani, N. (2010). Assessing the quality of scientific conferences based on bibliographic citations. Scientometrics, 83(1), 133–155. https://doi.org/10.1007/s11192-009-0078-y.
Meho, L. I., & Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar. Journal of the American Society for Information Science and Technology, 58(13), 2105–2125. https://doi.org/10.1002/asi.20677.
Michels, C., & Fu, J.-Y. (2014). Systematic analysis of coverage and usage of conference proceedings in web of science. Scientometrics, 100(2), 307–327. https://doi.org/10.1007/s11192-014-1309-4.
Montesi, M., & Owen, J. M. (2008). From conference to journal publication: How conference papers in software engineering are extended for publication in journals. Journal of the American Society for Information Science and Technology, 59(5), 816–829. https://doi.org/10.1002/asi.20805.
Nomaler, Ö., Frenken, K., & Heimeriks, G. (2013). Do more distant collaborations have more citation impact? Journal of Informetrics, 7(4), 966–971. https://doi.org/10.1016/j.joi.2013.10.001.
Onodera, N., & Yoshikane, F. (2015). Factors affecting citation rates of research articles. Journal of the Association for Information Science and Technology, 66(4), 739–764. https://doi.org/10.1002/asi.23209.
Peng, T.-Q., & Zhu, J. J. H. (2012). Where you publish matters most: A multilevel analysis of factors affecting citations of internet studies. Journal of the American Society for Information Science and Technology, 63(9), 1789–1803. https://doi.org/10.1002/asi.22649.
Radicchi, F., & Castellano, C. (2013). Analysis of bibliometric indicators for individual scholars in a large data set. Scientometrics, 97(3), 627–637. https://doi.org/10.1007/s11192-013-1027-3.
Rahm, E., & Thor, A. (2005). Citation analysis of database publications. SIGMOD Record, 34(4), 48–53. https://doi.org/10.1145/1107499.1107505.
Sakr, S., & Alomari, M. (2012). A decade of database conferences: A look inside the program committees. Scientometrics, 91(1), 173–184. https://doi.org/10.1007/s11192-011-0530-7.
Scopus. (2013). Content coverage guide. Retrieved from https://files.sciverse.com/documents/pdf/ContentCoverageGuide-jan-2013.pdf.
Shirakawa, N., Furukawa, T., Nomura, M., & Okuwada, K. (2012). Global competition and technological transition in electrical, electronic, information and communication engineering: Quantitative analysis of periodicals and conference proceedings of the IEEE. Scientometrics, 91(3), 895–910. https://doi.org/10.1007/s11192-011-0566-8.
Sin, S.-C. J. (2011). International coauthorship and citation impact: A bibliometric study of six LIS journals, 1980–2008. Journal of the American Society for Information Science and Technology, 62(9), 1770–1783. https://doi.org/10.1002/asi.21572.
Song, M., Heo, G. E., & Kim, S. Y. (2014). Analyzing topic evolution in bioinformatics: Investigation of dynamics of the field with conference data in DBLP. Scientometrics, 101(1), 397–428.
Souto, M. A. M., Warpechowski, M., & de Oliveira, J. P. M. (2007). An ontological approach for the quality assessment of computer science conferences. Berlin: Springer.
Tahamtan, I., Safipour Afshar, A., & Ahamdzadeh, K. (2016). Factors affecting number of citations: A comprehensive review of the literature. Scientometrics, 107(3), 1195–1225. https://doi.org/10.1007/s11192-016-1889-2.
Thelwall, M., & Wilson, P. (2014). Regression for citation data: An evaluation of different methods. Journal of Informetrics, 8(4), 963–971. https://doi.org/10.1016/j.joi.2014.09.011.
Vanclay, J. K. (2013). Factors affecting citation rates in environmental science. Journal of Informetrics, 7(2), 265–271. https://doi.org/10.1016/j.joi.2012.11.009.
Vardi, M. Y. (2010). Revisiting the publication culture in computing research. Communications of the ACM, 53(3), 5. https://doi.org/10.1145/1666420.1666421.
Vasilescu, B., Serebrenik, A., Mens, T., van den Brand, M. G. J., & Pek, E. (2014). How healthy are software engineering conferences? Science of Computer Programming, 89, 251–272. https://doi.org/10.1016/j.scico.2014.01.016.
Vrettas, G., & Sanderson, M. (2015). Conferences versus journals in computer science. Journal of the Association for Information Science and Technology, 66(12), 2674–2684. https://doi.org/10.1002/asi.23349.
Wainer, J., Eckmann, M., & Rocha, A. (2015). Peer-selected “best papers”—Are they really that “good”? PLoS ONE, 10(3), e0118446. https://doi.org/10.1371/journal.pone.0118446.
Wainer, J., Przibisczki de Oliveira, H., & Anido, R. (2011). Patterns of bibliographic references in the ACM published papers. Information Processing and Management, 47(1), 135–142. https://doi.org/10.1016/j.ipm.2010.07.002.
Wainer, J., & Valle, E. (2013). What happens to computer science research after it is published? Tracking CS research lines. Journal of the American Society for Information Science and Technology, 64(6), 1104–1111.
Waltman, L. (2016). A review of the literature on citation impact indicators. Journal of Informetrics, 10(2), 365–391. https://doi.org/10.1016/j.joi.2016.02.007.
Wersig, G. (1993). Information science: The study of postmodern knowledge usage. Information Processing and Management, 29(2), 229–239. https://doi.org/10.1016/0306-4573(93)90006-Y.
Wuehrer, G. A., & Smejkal, A. E. (2013). The knowledge domain of the academy of international business studies (AIB) conferences: A longitudinal scientometric perspective for the years 2006–2011. Scientometrics, 95(2), 541–561. https://doi.org/10.1007/s11192-012-0909-0.
Zahedi, Z., Costas, R., & Wouters, P. (2014). How well developed are altmetrics? A cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications. Scientometrics, 101(2), 1491–1513. https://doi.org/10.1007/s11192-014-1264-0.
Zhang, Y., & Jia, X. (2013). Republication of conference papers in journals? Learned Publishing, 26(3), 189–196. https://doi.org/10.1087/20130307.
Acknowledgements
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean government (Ministry of Science and ICT) (NRF-2018R1C1B6002434).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lee, D.H. Predictive power of conference-related factors on citation rates of conference papers. Scientometrics 118, 281–304 (2019). https://doi.org/10.1007/s11192-018-2943-z
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-018-2943-z