Marta Catillo
Umberto Villano
ABSTRACT
The availability of ready-to-use public security datasets is fostering measurement-driven research by a wide community of academics and practitioners. Recent trends in this area put forth a substantial body of literature on anomaly and attack detection on the top of public labelled datasets. Much of this literature blindly reuses existing datasets by overlooking the cybersecurity facets of the network traffic therein, in terms of its real impact on service availability and performance of operations.
This paper addresses the representativeness of network traffic data provided by public datasets for cybersecurity research. To this aim, it proposes an initial exploration of the topic by means of a case study on Denial of Service (DoS) traffic of CICIDS2017, which is a recent dataset collected in a controlled environment that gained massive attention over the past two years. DoS traffic, which is available in CICIDS2017 in the form of packet data files, is replayed against a victim server in a controlled testbed. Measurements indicate that the DoS traffic, although somewhat relevant at network-level, has limited impact at application-level (i.e., by taking into account the performance of the victim under attack). The findings provide some key insights into the limitations of the data assessed in the study, paving the way for the construction of more rigorous datasets conceived with a multilayer perspective and that reflect actual traffic conditions under normative operations and disruptive attacks.
- 1999. KDD Cup Data. http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.htmlGoogle Scholar
- A. Ahmim, L. A. Maglaras, M. A. Ferrag, M. Derdour, and H. Janicke. 2019. A Novel Hierarchical Intrusion Detection System Based on Decision Tree and Rules-Based Models. In International Conference on Distributed Computing in Sensor Systems. IEEE, 228--233.Google Scholar
- D. Aksu and M. Ali Aydin. 2018. Detecting Port Scan Attempts with Comparative Analysis of Deep Learning and Support Vector Machine Algorithms. In International Congress on Big Data, Deep Learning and Fighting Cyber Terrorism. IEEE, 77--80.Google Scholar
- M. Alizadeh, A. Greenberg, D. A. Maltz, J. Padhye, P. Patel, B. Prabhakar, S. Sengupta, and M. Sridharan. 2010. Data center TCP (DCTCP). ACM SIGCOMM Comput. Commun. Rev. 40, 4 (2010), 63--74.Google ScholarDigital Library
- M. Catillo, M. Rak, and U. Villano. 2019. Discovery of DoS attacks by the ZED-IDS anomaly detector. Journal of High Speed Networks 25 (2019), 349--365.Google ScholarCross Ref
- M. Catillo, M. Rak, and U. Villano. 2020. 2L-ZED-IDS: A Two-Level Anomaly Detector for Multiple Attack Classes. In Web, Artificial Intelligence and Network Applications. WAINA2020. (Advances in Intelligent Systems and Computing), L. Barolli, F. Amato, F. Moscato, T. Enokido, and M. Takizawa (Eds.). Springer, Cham, 687--696.Google Scholar
- D. Cotroneo, A. Paudice, and A. Pecchia. 2019. Empirical Analysis and Validation of Security Alerts Filtering Techniques. IEEE Transactions on Dependable and Secure Computing 16, 5 (2019), 856--870.Google ScholarCross Ref
- F. S. de Lima Filho, F. A. F. Silveira, A. de Medeiros Brito Júnior, G. Vargas-Solar, and L. F. Silveira. 2019. Smart Detection: An Online Approach for DoS/DDoS Attack Detection Using Machine Learning. Security and Communication Networks 1574749 (2019), 1--15.Google ScholarCross Ref
- U. Franke and J. Brynielsson. 2014. Cyber situational awareness - A systematic review of the literature. Comput. Secur. 46 (2014), 18 -- 31.Google ScholarCross Ref
- R. Jain. 1991. The Art of Computer Systems Performance Analysis. J. Wiley & Sons New York.Google Scholar
- A. Khraisat, I. Gondal, P. Vamplew, and J. Kamruzzaman. 2019. Survey of intrusion detection systems: techniques, datasets and challenges. Cybersecur 2 (2019), 20.Google ScholarCross Ref
- D. Kshirsagar and S. Kumar. 2020. Identifying Reduced Features Based on IG-Threshold for DoS Attack Detection Using PART. In Distributed Computing and Internet Technology, D. V. Hung and M. D'Souza (Eds.). Springer, Cham, 411--419.Google Scholar
- H. Liu and B. Lang. 2019. Machine Learning and Deep Learning Methods for Intrusion Detection Systems: A Survey. Applied Sciences 9, 20 (2019), 4396.Google ScholarCross Ref
- G. Maciá-Fernández, J. Camacho, R. Magán-Carrión, P. García-Teodoro, and R. Therón. 2017. UGR'16: A new dataset for the evaluation of cyclostationarity-based network IDSs. Comput. Secur. 73 (2017), 411 -- 424.Google ScholarCross Ref
- G. Mantas, N. Stakhanova, H. Gonzalez, H. Jazi, and A. Ghorbani. 2015. Application-layer denial of service attacks: Taxonomy and survey. International Journal of Information and Computer Security 7 (2015), 216 -- 239.Google ScholarDigital Library
- N. Martins, J. M. Cruz, T. Cruz, and P. H. Abreu. 2019. Analyzing the Footprint of Classifiers in Adversarial Denial of Service Contexts. In Progress in Artificial Intelligence, P. M. Oliveira, P. Novais, and L. Reis (Eds.). Springer, Cham, 256--267.Google Scholar
- J. McHugh. 2000. Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory. ACM Transactions on Information and System Security 3, 4 (2000), 262--294.Google ScholarDigital Library
- N. Moustafa and J. Slay. 2015. UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In Military Communications and Information Systems Conference. IEEE, 1--6.Google Scholar
- R. Panigrahi and S. Borah. 2018. A detailed analysis of CICIDS2017 dataset for designing Intrusion Detection Systems. International Journal of Engineering & Technology 7, 3.24 (2018).Google Scholar
- X. Peng, W. Huang, and Z. Shi. 2019. Adversarial Attack Against DoS Intrusion Detection: An Improved Boundary-Based Method. In International Conference on Tools with Artificial Intelligence. IEEE, 1288--1295.Google Scholar
- X. Qu, L. Yang, K. Guo, L. Ma, T. Feng, S. Ren, and M. Sun. 2019. Statistics-enhanced Direct Batch Growth Self-organizing Mapping for efficient DoS Attack Detection. IEEE Access 7 (2019), 78434--78441.Google ScholarCross Ref
- E. Raftopoulos and X. Dimitropoulos. 2013. Understanding Network Forensics Analysis in an Operational Environment. In Security and Privacy Workshops. IEEE, 111--118.Google Scholar
- M. Ring, S. Wunderlich, D. Scheuring, D. Landes, and A. Hotho. 2019. A Survey of Network-based Intrusion Detection Data Sets. Comput. Secur. 86 (2019), 147--167.Google ScholarCross Ref
- I. Sharafaldin, A. H. Lashkari, and A. A. Ghorbani. 2018. Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization. In International Conference on Information Systems Security and Privacy. SciTePress, 108--116.Google Scholar
- A. Sharma, Z. Kalbarczyk, J. Barlow, and R. Iyer. 2011. Analysis of security data from a large computing organization. In International Conference on Dependable Systems Networks. IEEE, 506--517.Google Scholar
- A. Shiravi, H. Shiravi, M. Tavallaee, and A. Ghorbani. 2012. Toward developing a systematic approach to generate benchmark datasets for intrusion detection. Comput. Secur. 31 (2012), 357--374.Google ScholarDigital Library
- S. T. Zargar, J. Joshi, and D. Tipper. 2013. A Survey of Defense Mechanisms Against Distributed Denial of Service (DDoS) Flooding Attacks. IEEE Communications Surveys & Tutorials 15 (2013), 2046 -- 2069.Google ScholarCross Ref
- M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani. 2009. A detailed analysis of the KDD CUP 99 data set. In Symposium on Computational Intelligence for Security and Defense Applications. IEEE, 1--6.Google Scholar
- S. Wankhede and D. Kshirsagar. 2018. DoS Attack Detection Using Machine Learning and Neural Network. In International Conference on Computing Communication Control and Automation. IEEE, 1--5.Google Scholar
- C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and A. Wesslén. 2000. Experimentation in Software Engineering: An Introduction. Kluwer Academic.Google ScholarDigital Library
Index Terms
- A case study on the representativeness of public DoS network traffic data for cybersecurity research
Recommendations
Demystifying the role of public intrusion datasets: A replication study of DoS network traffic data
AbstractPublic intrusion datasets are contributing to make security research accessible to a large community of users, but are often trusted and reused neglecting the actual impact of the attacks therein on victim services. This paper ...
Measurement-Based Analysis of a DoS Defense Module for an Open Source Web Server
Testing Software and SystemsAbstractDenial of Service (DoS) attacks represent an ever evolving landscape, which ranges from bruteforce flooding approaches to more sophisticated low-bandwidth slow techniques. DoS has become a major threat to the availability of modern web servers ...
Research on DoS attack and detection programming
IITA'09: Proceedings of the 3rd international conference on Intelligent information technology applicationThe DoS attack is the most popular attack in the network security with the development of network and internet. In this paper, the DoS attack principle is discussed and some DoS attack methods are deeply analyzed. The DoS attack detection technologies ...
Comments