Abstract
The understanding of the inner workings of a research community is essential to evaluate the impact of an author as well as to decide where and how to publish results. One of the key metrics is the number of citations that a publication receives. In parallel, information security is now a key and strategic area, partially fueled by the advent of the Internet of Things (IoT) and the need of pursuing cybercriminals by using digital forensics techniques. Therefore, this paper analyzes several factors influencing the number of citations in the domain of information security, such as differences between journal and conference publications, or the impact of the number of pages and the length of the abstract. To obtain quantitative results, we investigated papers of six sub-disciplines, i.e., anonymity and privacy, cryptography, information hiding, IoT and Cyber-Physical System security, digital forensics and incident response, and network security. For each sub-domain, we used metadata of 5000 publications collected from IEEE-Xplore. Results indicate some clear behaviors, for instance, papers tend to receive more citations when their abstract is longer and the number of references positively influences the performance of the work.
Similar content being viewed by others
Notes
Fernandes and Monteiro reported a continuous per-decade increase in the number of authors per paper in information security research since the mid-1970s (Fernandes and Monteiro 2017). As a side-effect of our analysis, were able to confirm their observation.
References
Anderson, T. (2009). Conference reviewing considered harmful. ACM SIGOPS Operating Systems Review, 43(2), 108–116. https://doi.org/10.1145/1531793.1531815.
Baldwin, J., Alhawi, O. M., Shaughnessy, S., Akinbi, A., & Dehghantanha, A. (2018). Emerging from the cloud: A bibliometric analysis of cloud forensics studies. In A. Dehghantanha, M. Conti, & T. Dargahi (Eds.), Cyber threat intelligence (pp. 311–331). Cham: Springer.
Bornmann, L., & Leydesdorff, L. (2018). Count highly-cited papers instead of papers with h citations: Use normalized citation counts and compare “like with like”!. Scientometrics, 115(2), 1119–1123. https://doi.org/10.1007/s11192-018-2682-1.
Burcham, M., Al-Zyoud, M., Carver, J. C., Alsaleh, M., Du, H., Gilani, F., Jiang, J., Rahman, A., Kafalı, O., Al-Shaer, E., & Williams, L. (2017). Characterizing scientific reporting in security literature: An analysis of ACM CCS and IEEE S&P papers. In Proceedings of the hot topics in science of security: Symposium and bootcamp, HoTSoS (pp. 13–23). ACM, New York. https://doi.org/10.1145/3055305.3055307
Cabaj, K., Caviglione, L., Mazurczyk, W., Wendzel, S., Woodward, A., & Zander, S. (2018). The new threats of information hiding: the road ahead. IT Professional, 20(3), 31–39.
Carver, J. C., Burcham, M., Kocak, S. A., Bener, A., Felderer, M., Gander, M., King, J., Markkula, J., Oivo, M., Sauerwein, C., & Williams, L. (2016). Establishing a baseline for measuring advancement in the science of security: An analysis of the 2015 IEEE security and privacy proceedings. In Proceedings of the symposium and bootcamp on the science of security, HotSos’16 (pp. 38–51). ACM, New York. https://doi.org/10.1145/2898375.2898380
Cavelty, M. D. (2018). Cybersecurity research meets science and technology studies. Politics and Governance, 6(2), 22–30.
Chakraborty, J., Pradhan, D., Dutta, H. S., Nandi, S., & Chakraborty, T. (2018). On good and bad intentions behind anomalous citation patterns among journals in computer sciences. CoRR arXiv:1807.10804
Chakraborty, T. (2018). Role of interdisciplinarity in computer sciences: Quantification, impact and life trajectory. Scientometrics, 114(3), 1011–1029. https://doi.org/10.1007/s11192-017-2628-z.
Cobo, M., Jürgens, B., Herrero-Solana, V., Martínez, M., & Herrera-Viedma, E. (2018). Industry 4.0: A perspective based on bibliometric analysis. Procedia Computer Science, 139, 364–371. https://doi.org/10.1016/j.procs.2018.10.278. (6th International Conference on Information Technology and Quantitative Management).
Collberg, C., & Proebsting, T. A. (2016). Repeatability in computer systems research. Communications of the ACM, 59(3), 62–69. https://doi.org/10.1145/2812803.
Corporation, M. (2010). The science of cyber security. Tech. Rep. JSR-10-102, MITRE. http://www.fas.org/irp/agency/dod/jason/cyber.pdf
Fernandes, J. M., & Monteiro, M. P. (2017). Evolution in the number of authors of computer science publications. Scientometrics, 110, 529–539.
Fiala, D., & Tutoky, G. (2017). Computer science papers in web of science: A bibliometric analysis. Publications, 5(4), 23. https://doi.org/10.3390/publications5040023.
Goodrum, A. A., McCain, K. W., Lawrence, S., & Giles, C. L. (2001). Scholarly publishing in the internet age: A citation analysis of computer science literature. Information Processing & Management, 37(5), 661–675.
Herley, C., & van Oorschot, P. C. (2017). Sok: Science, security and the elusive goal of security as a scientific pursuit. In 2017 IEEE symposium on security and privacy (SP) (pp. 99–120). IEEE. https://doi.org/10.1109/SP.2017.38
Humayed, A., Lin, J., Li, F., & Luo, B. (2017). Cyber-physical systems security-a survey. IEEE Internet of Things Journal, 4(6), 1802–1831.
Iqbal, W., Qadir, J., Tyson, G., Mian, A. N., Hassan, Su, & Crowcroft, J. (2019). A bibliometric analysis of publications in computer networking research. Scientometrics, 119(2), 1121–1155. https://doi.org/10.1007/s11192-019-03086-z.
Isenberg, P., Isenberg, T., Sedlmair, M., Chen, J., & Möller, T. (2017). Visualization as seen through its research paper keywords. IEEE Transactions on Visualization and Computer Graphics, 23(1), 771–780. https://doi.org/10.1109/TVCG.2016.2598827.
Jia, H., & Saule, E. (2017). An analysis of citation recommender systems: Beyond the obvious. In Proceedings of the 2017 IEEE/ACM international conference on advances in social networks analysis and mining 2017, ASONAM ’17 (pp. 216–223). ACM, New York, NY. https://doi.org/10.1145/3110025.3110150
Kott, A. (2014). Towards fundamental science of cyber security. In R. E. Pino (Ed.), Network science and cybersecurity (pp. 1–13). New York: Springer.
Kousha, K., & Thelwall, M. (2007). Google scholar citations and google web/url citations: A multi-discipline exploratory analysis. Journal of the Association for Information Science and Technology, 58(7), 1055–1065.
Li, M. (2018). Classifying and ranking topic terms based on a novel approach: Role differentiation of author keywords. Scientometrics, 116(1), 77–100.
MacDonald, K. I., & Dressler, V. (2018). Using citation analysis to identify research fronts: A case study with the internet of things. Science & Technology Libraries, 37(2), 171–186. https://doi.org/10.1080/0194262X.2017.1415183.
Makawana, P. R., & Jhaveri, R. H. (2018). A bibliometric analysis of recent research on machine learning for cyber security. In Y. C. Hu, S. Tiwari, K. K. Mishra, & M. C. Trivedi (Eds.), Intelligent Communication and Computational Technologies (pp. 213–226). Singapore: Springer Singapore.
Mazurczyk, W., & Caviglione, L. (2015). Information hiding as a challenge for malware detection. IEEE Security Privacy, 13(2), 89–93.
Meyer, B. (2011). The nastiness problem in computer science. Posting in the blog of Comm. ACM. http://cacm.acm.org/blogs/blog-cacm/123611-the-nastiness-problem-in-computer-science/fulltext
Mogul, J. C. (2013). Towards more constructive reviewing of CS papers. ACM SIGCOMM Computer Communication Review, 43(3), 90–94. https://doi.org/10.1145/2500098.2500112.
Muhamedyev, R. I., Aliguliyev, R. M., Shokishalov, Z. M., & Mustakayev, R. R. (2018). New bibliometric indicators for prospectivity estimation of research fields. Annals of Library and Information Studies, 65, 62–69.
Shen, S., Rousseau, R., & Wang, D. (2018). Do papers with an institutional e-mail address receive more citations than those with a non-institutional one? Scientometrics, 115, 1039–1050.
Stanton, J., Mastrangelo, P., Stam, K., & Jolton, J. (2004). Behavioral information security: Two end user survey studies of motivation and security practices. In AMCIS 2004 proceedings (pp. 175).
Sun, X., Ding, K., & Lin, Y. (2016). Mapping the evolution of scientific fields based on cross-field authors. Journal of Informetrics, 10(3), 750–761.
Vieira, E. S., & Gomes, J. A. (2009). Citations to scientific articles: Its distribution and dependence on the article features. Journal of Informetrics, 4, 1–13.
Wendzel, S. (2018). Get me cited, Scotty! Analysis of academic publications in covert channel/steganography research. In Proceedings of the 13th international conference on availability, reliability and security (ARES’18) (pp. 13:1–13:8). ACM, New York, NY. https://doi.org/10.1145/3230833.3233265
Yan, B. N., Lee, T. S., & Lee, T. P. (2015). Mapping the intellectual structure of the internet of things (IoT) field (2000–2014): a co-word analysis. Scientometrics, 105(2), 1285–1300.
Yang, K., & Meho, L. I. (2006). Citation analysis: A comparison of google scholar, scopus, and web of science. Proceedings of the Association for Information Science and Technology, 43(1), 1–15.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wendzel, S., Lévy-Bencheton, C. & Caviglione, L. Not all areas are equal: analysis of citations in information security research. Scientometrics 122, 267–286 (2020). https://doi.org/10.1007/s11192-019-03279-6
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-019-03279-6