Skip to main content
SpringerLink
Log in

Big Data Processing for Intrusion Detection System Context: A Review

  • Conference paper
  • First Online:
Intelligent Systems Design and Applications (ISDA 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1181))

  • 641 Accesses

Abstract

The rapid growth of data, the increasing number of network based applications, and the advent of the omnipresence of internet and connected devices have affected the importance of information security. Hence, a security system such as an Intrusion Detection System (IDS) becomes a fundamental requirement. However, the complexity of the generated data and their huge size, plus, the variation of Cyber-attacks on: the network traffic, wireless network traffic, worldwide network traffic, connected devices and 5 G communication media, lead to hinder the IDS’s efficiency. Dealing with this huge amount of traffic is challenging and requires deploying new big data security solutions. This paper proposes an overview of intrusion detection which offers a review of IDS that deploy big data technologies and provides interesting recommendations for further study.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Vani, Y.K., Krishnamurthy: Survey anomaly detection in network using big data analytics. In: 2017 International Conference Energy Communication Data Analytics Soft Computing (ICECDS) (2017)

    Google Scholar 

  2. Sharma, P., Sengupta, J., Suri, P.: Survey of intrusion detection techniques and architectures. Cloud Comput. Int. J. High Perform. Comput. Netw. 13, 184 (2019)

    Google Scholar 

  3. Ariyaluran Habeeb, R., Nasaruddin, F., Gani, A., Targio Hashem, I., Ahmed, E., Imran, M.: Real-time big data processing for anomaly detection: a survey. Int. J. Inform. Manag. 45, 289–307 (2019)

    Article  Google Scholar 

  4. Provider, S., Forecasts, V., Papers, W.: Cisco visual networking index: forecast and trends, White Paper (2017–2022). https://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/white-paper-c11–741490.html

  5. Resende, P.A.A., Drummond, A.C.: A survey of random forest based methods for intrusion detection systems. ACM Comput. Surv. 51, 1–36 (2018)

    Article  Google Scholar 

  6. Chaabouni, N., Mosbah, M., Zemmari, A., Sauvignac, C., Faruki, P.: Network intrusion detection for IoT security based on learning techniques. IEEE Commun. Surv. Tutorials 21(3), 2671–2701 (2019)

    Article  Google Scholar 

  7. Sheenam, S., Dhiman, S.: Comprehensive review: intrusion detection system and techniques. IOSR J. Comput. Eng. 18, 20–25 (2016)

    Article  Google Scholar 

  8. Bostami, B., Ahmed, M.: Intrusion detection for big data. Data Anal. 375–402 (2018)

    Google Scholar 

  9. Zuech, R., Khoshgoftaar, T.M., Wald, R.: Intrusion detection and big heterogeneous data: a survey. J. Big Data 2(1), 3 (2015). https://doi.org/10.1186/s40537-015-0013-4

    Article  Google Scholar 

  10. Patel, A., Taghavi, M., Bakhtiyari, K., Celestino Júnior, J.: An intrusion detection and prevention system in cloud computing: a systematic review. J. Netw. Comput. Appl. 36, 25–41 (2013)

    Article  Google Scholar 

  11. Masarat, S., Sharifian, S., Taheri, H.: Modified parallel random forest for intrusion detection systems. J. Supercomput. 72, 2235–2258 (2016)

    Article  Google Scholar 

  12. Liao, H.J., Lin, C.H.R., Lin, Y.C., Tung, K.Y.: Intrusion detection system: a comprehensive review. J. Netw. Comput. Appl. 36, 16–24 (2013)

    Article  Google Scholar 

  13. Canali, D., Cova, M., Vigna, G., Kruegel, C.: Prophiler. In: Proceedings of the 20th International Conference World Wide Web - WWW 2011 (2011)

    Google Scholar 

  14. Shiravi, A., Shiravi, H., Tavallaee, M., Ghorbani, A.A.: Toward developing a systematic approach to generate benchmark datasets for intrusion detection. Comput. Secur. 31, 357–374 (2012)

    Article  Google Scholar 

  15. Moustafa, N., Slay, J.: UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In: 2015 Military Communications and Information Systems Conference (MilCIS) (2015)

    Google Scholar 

  16. Hafsa, M., Jemili, F.: Comparative study between big data analysis techniques in intrusion detection. Big Data Cogn. Comput. 3, 1 (2018)

    Article  Google Scholar 

  17. Essid, M., Jemili, F.: Combining intrusion detection datasets using MapReduce. In: IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 4724–4728 (2016)

    Google Scholar 

  18. Elayni, M., Jemili, F.: Using mongodb databases for training and combining intrusion detection datasets. In: Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, pp. 17–29 (2017). https://doi.org/10.1007/978-3-319-62048-0_2

  19. Guo, K., Xu, T., Kui, X., Zhang, R., Chi, T.: Towards efficient intelligence fusion for deep learning from real-time and heterogeneous data. Inform. Fusion 51, 215–223 (2019)

    Article  Google Scholar 

  20. Lv, K., Chen, Y., Hu, C.: Dynamic defense strategy against advanced persistent threat under heterogeneous networks. Inform. Fusion 49, 216–226 (2019)

    Article  Google Scholar 

  21. Shanbhogue, R.D., Beena, B.M.: Survey of data mining (DM) and machine learning (ML) methods on cyber security. J. Sci. Technol. 10, 1–7 (2017)

    Google Scholar 

  22. Zhao, R., Yan, R., Chen, Z., Mao, K., Wang, P., Gao, R.X.: Deep learning and its applications to machine health monitoring: a survey. arXiv preprint arXiv:1612.07640 (2016)

  23. Ahmad, S., Lavin, A., Purdy, S., Agha, Z.: Unsupervised real-time anomaly detection for streaming data. Neurocomputing 262, 134–147 (2017)

    Article  Google Scholar 

  24. Natesan, P., Rajalaxmi, R., Gowrison, G., Balasubramanie, P.: Hadoop based parallel binary bat algorithm for network intrusion detection. Int. J. Parallel Program. 45, 1194–1213 (2016)

    Article  Google Scholar 

  25. Mehta, S., Kothuri, P., Garcia, D.L.: A big data architecture for log data storage and analysis. Integr. Intell. Comput. Commun. Secur. Stud. Comput. Intell. 201–209 (2018)

    Google Scholar 

  26. Dahiya, P., Srivastava, D.: Network intrusion detection in big dataset using spark. Procedia Comput. Sci. 132, 253–262 (2018)

    Article  Google Scholar 

  27. Othman, S., Ba-Alwi, F., Alsohybe, N., Al-Hashida, A.: Intrusion detection model using machine learning algorithm on big data environment. J. Big Data 5(1), 1–12 (2018)

    Article  Google Scholar 

  28. Marchal, S., Jiang, X., State, R., Engel, T.: A big data architecture for large scale security monitoring. In: 2014 IEEE International Congress Big Data (2014)

    Google Scholar 

  29. Viegas, E., Santin, A., Bessani, A., Neves, N.: BigFlow: real-time and reliable anomaly-based intrusion detection for high-speed networks. Future Gener. Comput. Syst. 93, 473–485 (2019)

    Article  Google Scholar 

  30. Yan, Z., Liu, J., Yang, L.T., Pedrycz, W.: Data fusion in heterogeneous networks. Inform. Fusion 53, 1–3 (2020)

    Article  Google Scholar 

  31. Cyber security: how Cisco uses graph analytics to identify threats. https://linkurio.us/blog/cyber-security

  32. Meng, W., Tischhauser, E.W., Wang, Q., Wang, Y., Han, J.: When intrusion detection meets blockchain technology: a review. IEEE Access 6, 10179–10188 (2018)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ISITCom, MARS Research Laboratory, LR17ES05, Universite de Sousse, Sousse, Tunisia

    Marwa Elayni, Farah Jemili & Ouajdi Korbaa

  2. ITI Laboratory, IMT Atlantique, 29238, Brest, France

    Basel Solaiman

Authors
  1. Marwa Elayni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Farah Jemili
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ouajdi Korbaa
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Basel Solaiman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marwa Elayni .

Editor information

Editors and Affiliations

  1. Scientific Network for Innovation and Research Excellence, Machine Intelligence Research Labs (MIR), Auburn, WA, USA

    Ajith Abraham

  2. Université Paris-Est Créteil Val-de-Ma, Creteil Cedex, France

    Patrick Siarry

  3. School of Information Science and Engineering, University of Jinan, Jinan, Shandong, China

    Kun Ma

  4. Department of Construction Management and Real Estate, Vilnius Gediminas Technical University, Vilnius, Lithuania

    Arturas Kaklauskas

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Elayni, M., Jemili, F., Korbaa, O., Solaiman, B. (2021). Big Data Processing for Intrusion Detection System Context: A Review. In: Abraham, A., Siarry, P., Ma, K., Kaklauskas, A. (eds) Intelligent Systems Design and Applications. ISDA 2019. Advances in Intelligent Systems and Computing, vol 1181. Springer, Cham. https://doi.org/10.1007/978-3-030-49342-4_12

Download citation

Publish with us

Policies and ethics

Search

Navigation