ABSTRACT
The paper proposes an approach to security information processing in order to detect computer attacks and network anomalies based on big data technologies. The main contribution of the work is in the development, implementation and investigation of the proposed combined framework for processing security data using parallel computing environment and measuring the performance of the implemented system for detection of network attacks and anomalies. The research goal is to increase the performance of attack detection (under the given requirements for accuracy of solutions) compared to the traditional IDS application. The implemented approach is built using the open source systems Snort and Spark. The paper discusses the capabilities and performance assessment of parallel data processing in order to detect computer attacks and network anomalies, as well as key principles of working with big data. The presented main results of an experimental performance evaluation of the applied approach confirm its high efficiency for analyzing network traffic and security events.
- R. Patgiri, A. Ahmed. 2016. Big Data: The V's of the Game Changer Paradigm. 18th IEEE International Conference on High Performance Computing and Communications, 17--24.Google ScholarCross Ref
- I. Kotenko, I. Saenko, A. Kushnerevich. 2017. Parallel big data processing system for security monitoring in Internet of Things networks. Journal of Wireless Mobile Networks, Ubiquitous Computing, and Dependable Applications (JoWUA), 8 (4), pp. 60--74.Google Scholar
- I. Kotenko, A. Kuleshov, and I. Ushakov. 2017. Aggregation of elastic stack instruments for collecting, storing and processing of security information and events. Proc. of the 14th IEEE Conference on Advanced Trusted Computing (ATC). San Francisco, CA, USA, pp.1550--1557.Google Scholar
- I. Kotenko, I. Saenko, A. Branitskiy. Framework for Mobile Internet of Things Security Monitoring based on Big Data Processing and Machine Learning. IEEE Access, 2018, Vol.6. pp.72714--72723.Google ScholarCross Ref
- A.A. Cardenas, P.K. Manadhata, and S.P. Rajan. 2013. Big data analytics for security. IEEE Security & Privacy, 11(6), 74--76.Google ScholarDigital Library
- P.G. Prathibha, E.D. Dileesh. 2013. Design of a Hybrid Intrusion Detection System using Snort and Hadoop. International Journal of Computer Applications. P. 1--6.Google Scholar
- J.J. Cheon, T.-Y. Choe. 2013. Distributed Processing of Snort Alert Log using Hadoop. International Journal of Engineering and Technology, 1--7.Google Scholar
- C.E. Otero, A. Peter. 2015. Research directions for engineering big data analytics software. IEEE Intelligent Systems, 30(1), 13--19.Google ScholarDigital Library
- M.N. Goncharova. 2014. Mechanism of parallel Big Data analysis in document-oriented DBMSs. Vestnik magistratury - Gerald of magistracy, vol.6--1 (33), 52--56 (In Russian).Google Scholar
- I.G. Bugrova, Yu. K. Demyanovich. 2011. Parallel computing algorithms and programming. Lecture course. SPbSU, 206 p. (in Russian)Google Scholar
- A.S. Shaburov, A.A. Mironov. 2015. Detection of computer attacks based on the functional approach. Bulletin of Perm University, 4 (31), 110--115. (in Russian)Google Scholar
- A.E. Yankovskaya, A.V. Yamshanov. 2016. Development of Cross-Platform Cognitive Tools Invariant to Problem Areas and their Integration into Intelligent Systems. Key Engineering Materials. Multifunctional Materials: Development and Application, Vol. 683, 609--616.Google Scholar
- D. Borthakur. 2007. The Hadoop Distributed File System: Architecture and Design. The Apache Software Foundation.Google Scholar
- M. Sharma, V. Chauhan. 2016. A review: MapReduce and Spark for big data analytics. International Journal of Advanced Technology in Engineering and Science, Vol. 4, 42--50.Google Scholar
- S. Lakavath, R. Naik. 2014. A Big Data Hadoop Architecture for Online Analysis. International Journal of Computer Science and Information Technology & Security (IJCSITS), Vol.4, No.6.Google Scholar
- H S. Bhosale1, D. P. Gadekar. 2014. A Review Paper on Big Data and Hadoop. International Journal of Scientific and Research Publications, 4(10).Google Scholar
- M.H. Padgavankar, S.R.Gupta. 2014. Big Data Storage and Challenges, M.H.Padgavankar. International Journal of Computer Science and Information Technologies (IJCSIT), 5 (2), 2218--2223.Google Scholar
- M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauley, M. Franklin, S. Shenker, and I. Stoica. 2011. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. Technical Report UCB/EECS-2011-82, EECS Department, University of California, Berkeley.Google Scholar
- I. Kotenko, A. Ulanov. 2006. Simulation of internet DDoS attacks and defense. Lecture Notes in Computer Science, Springer-Verlag, Vol.4176, pp. 327--342.Google Scholar
- I. Kotenko, M. Stepashkin, E. Doynikova. 2011. Security analysis of information systems taking into account social engineering attacks. Proceedings of 19th International Euromicro Conference on Parallel, Distributed, and Network-Based Processing (PDP 2011). IEEE, pp. 611--618.Google ScholarDigital Library
- I. Kotenko, E. Doynikova. 2015. The CAPEC based generator of attack scenarios for network security evaluation. The IEEE 8th International Conference on "Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications" (IDAACS'2015), pp.436--441.Google ScholarCross Ref
Index Terms
Combining spark and snort technologies for detection of network attacks and anomalies: assessment of performance for the big data framework
Recommendations
Study of snort-based IDS
ICWET '10: Proceedings of the International Conference and Workshop on Emerging Trends in TechnologyGeneral trend in industry is a shift from Intrusion Detection Systems (IDS) to Intrusion Prevention Systems (IPS). In this paper, we have investigated the motivations behind this trend. In addition, we have surveyed some of the available IDS/IPS tools. ...
Detecting, validating and characterizing computer infections in the wild
IMC '11: Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conferenceAlthough network intrusion detection systems (IDSs) have been studied for several years, their operators are still overwhelmed by a large number of false-positive alerts. In this work we study the following problem: from a large archive of intrusion ...
Data base support for intrusion detection with honeynets
TELE-INFO'07: Proceedings of the 6th WSEAS Int. Conference on Telecommunications and InformaticsAs computer attacks are becoming more and more difficult to identify the need for better and more efficient intrusion detection systems increases. The main problem with current intrusion detection systems is high rate of false alarms. In this paper we ...
Comments