skip to main content
10.1145/3357613.3357630acmotherconferencesArticle/Chapter ViewAbstractPublication PagessinConference Proceedingsconference-collections
research-article

Combining spark and snort technologies for detection of network attacks and anomalies: assessment of performance for the big data framework

Published:12 September 2019Publication History

ABSTRACT

The paper proposes an approach to security information processing in order to detect computer attacks and network anomalies based on big data technologies. The main contribution of the work is in the development, implementation and investigation of the proposed combined framework for processing security data using parallel computing environment and measuring the performance of the implemented system for detection of network attacks and anomalies. The research goal is to increase the performance of attack detection (under the given requirements for accuracy of solutions) compared to the traditional IDS application. The implemented approach is built using the open source systems Snort and Spark. The paper discusses the capabilities and performance assessment of parallel data processing in order to detect computer attacks and network anomalies, as well as key principles of working with big data. The presented main results of an experimental performance evaluation of the applied approach confirm its high efficiency for analyzing network traffic and security events.

References

  1. R. Patgiri, A. Ahmed. 2016. Big Data: The V's of the Game Changer Paradigm. 18th IEEE International Conference on High Performance Computing and Communications, 17--24.Google ScholarGoogle ScholarCross RefCross Ref
  2. I. Kotenko, I. Saenko, A. Kushnerevich. 2017. Parallel big data processing system for security monitoring in Internet of Things networks. Journal of Wireless Mobile Networks, Ubiquitous Computing, and Dependable Applications (JoWUA), 8 (4), pp. 60--74.Google ScholarGoogle Scholar
  3. I. Kotenko, A. Kuleshov, and I. Ushakov. 2017. Aggregation of elastic stack instruments for collecting, storing and processing of security information and events. Proc. of the 14th IEEE Conference on Advanced Trusted Computing (ATC). San Francisco, CA, USA, pp.1550--1557.Google ScholarGoogle Scholar
  4. I. Kotenko, I. Saenko, A. Branitskiy. Framework for Mobile Internet of Things Security Monitoring based on Big Data Processing and Machine Learning. IEEE Access, 2018, Vol.6. pp.72714--72723.Google ScholarGoogle ScholarCross RefCross Ref
  5. A.A. Cardenas, P.K. Manadhata, and S.P. Rajan. 2013. Big data analytics for security. IEEE Security & Privacy, 11(6), 74--76.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. P.G. Prathibha, E.D. Dileesh. 2013. Design of a Hybrid Intrusion Detection System using Snort and Hadoop. International Journal of Computer Applications. P. 1--6.Google ScholarGoogle Scholar
  7. J.J. Cheon, T.-Y. Choe. 2013. Distributed Processing of Snort Alert Log using Hadoop. International Journal of Engineering and Technology, 1--7.Google ScholarGoogle Scholar
  8. C.E. Otero, A. Peter. 2015. Research directions for engineering big data analytics software. IEEE Intelligent Systems, 30(1), 13--19.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. M.N. Goncharova. 2014. Mechanism of parallel Big Data analysis in document-oriented DBMSs. Vestnik magistratury - Gerald of magistracy, vol.6--1 (33), 52--56 (In Russian).Google ScholarGoogle Scholar
  10. I.G. Bugrova, Yu. K. Demyanovich. 2011. Parallel computing algorithms and programming. Lecture course. SPbSU, 206 p. (in Russian)Google ScholarGoogle Scholar
  11. A.S. Shaburov, A.A. Mironov. 2015. Detection of computer attacks based on the functional approach. Bulletin of Perm University, 4 (31), 110--115. (in Russian)Google ScholarGoogle Scholar
  12. A.E. Yankovskaya, A.V. Yamshanov. 2016. Development of Cross-Platform Cognitive Tools Invariant to Problem Areas and their Integration into Intelligent Systems. Key Engineering Materials. Multifunctional Materials: Development and Application, Vol. 683, 609--616.Google ScholarGoogle Scholar
  13. D. Borthakur. 2007. The Hadoop Distributed File System: Architecture and Design. The Apache Software Foundation.Google ScholarGoogle Scholar
  14. M. Sharma, V. Chauhan. 2016. A review: MapReduce and Spark for big data analytics. International Journal of Advanced Technology in Engineering and Science, Vol. 4, 42--50.Google ScholarGoogle Scholar
  15. S. Lakavath, R. Naik. 2014. A Big Data Hadoop Architecture for Online Analysis. International Journal of Computer Science and Information Technology & Security (IJCSITS), Vol.4, No.6.Google ScholarGoogle Scholar
  16. H S. Bhosale1, D. P. Gadekar. 2014. A Review Paper on Big Data and Hadoop. International Journal of Scientific and Research Publications, 4(10).Google ScholarGoogle Scholar
  17. M.H. Padgavankar, S.R.Gupta. 2014. Big Data Storage and Challenges, M.H.Padgavankar. International Journal of Computer Science and Information Technologies (IJCSIT), 5 (2), 2218--2223.Google ScholarGoogle Scholar
  18. M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauley, M. Franklin, S. Shenker, and I. Stoica. 2011. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. Technical Report UCB/EECS-2011-82, EECS Department, University of California, Berkeley.Google ScholarGoogle Scholar
  19. I. Kotenko, A. Ulanov. 2006. Simulation of internet DDoS attacks and defense. Lecture Notes in Computer Science, Springer-Verlag, Vol.4176, pp. 327--342.Google ScholarGoogle Scholar
  20. I. Kotenko, M. Stepashkin, E. Doynikova. 2011. Security analysis of information systems taking into account social engineering attacks. Proceedings of 19th International Euromicro Conference on Parallel, Distributed, and Network-Based Processing (PDP 2011). IEEE, pp. 611--618.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. I. Kotenko, E. Doynikova. 2015. The CAPEC based generator of attack scenarios for network security evaluation. The IEEE 8th International Conference on "Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications" (IDAACS'2015), pp.436--441.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Combining spark and snort technologies for detection of network attacks and anomalies: assessment of performance for the big data framework

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Other conferences
      SIN '19: Proceedings of the 12th International Conference on Security of Information and Networks
      September 2019
      179 pages
      ISBN:9781450372428
      DOI:10.1145/3357613

      Copyright © 2019 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 12 September 2019

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      Overall Acceptance Rate102of289submissions,35%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader