Abstract
Nowadays, with the immense amount of data that is circulating every second, the cybersecurity concerns are also growing. In recent years, cybersecurity-intrusion detection has become a very difficult research area in communication network and big data. Hence, traditional intrusion detection systems (IDSs) could not respond to the new security challenges. Therefore, IDSs require an effective and improved detection mechanism capable of detecting distributed intrusive activities and serious threats to network security. In this paper, we have proposed A Multi-Agent System (MAS), which is very suitable for IDSs as it meets the features required by the networks and Big Data issues, through cooperation, autonomy, and proactivity between agents to ensure the effective detection of intrusions without the intervention of an expert. Moreover, some experiments were conducted to evaluate the performance of our model in a Microsoft Azure Cloud, as it provides both processing power and storage capabilities using Apache Spark, and its Machine Learning Library (MLlib) to detect intrusions. A Random Forest algorithm is used to provide for the nature of the incoming data. Also, the use of the recent CSE-CIC-IDS2018 dataset will give better perspective about the system abilities against cyber-attacks. The results show that the proposed solution is much accurate than traditional intrusion detection systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sarker, I.H., Abushark, Y.B., Alsolami, F., Khan, A.I.: IntruDTree: A Machine Learning Based Cyber Security Intrusion Detection Model, p. 754. Symmetry. Multidisciplinary Digital Publishing Institute (2020)
Dounya, K., Okba, K., Hamza, S., Safa, S., Iman, H., Omar, B.: A new approach based mobile agent system for ensuring secure big data transmission and storage. In: 2017 International Conference on Mathematics and Information Technology (ICMIT), pp. 196–200. IEEE (2017)
Apache Spark. Apache Spark. [Online], 6 January 2021. https://spark.apache.org/
Apache Hadoop. Apache Hadoop. [Online] (2020). https://hadoop.apache.org/
Benyettou, N.: Modélisation des Systèmes Immunitaires Artificiel par les Systèmes Multi-Agents Pour la Détection d’intrusion dans les réseaux Informatique, 14 November 2017
Louati, F., Ktata, F.B.: A Deep Learning-Based Multi-agent System for Intrusion Detection, pp. 1–13. SN Applied Sciences. Springer (2020)
Mokhtari, S.M., Moulkhaloua, A.: Système DE Détection D’intrusions Informatiquespar Système Multi Agents (2018)
Ouiazzane, S., Addou, M., Barramou, F.: A multi-agent model for network intrusion detection. In: 2019 1st International Conference on Smart Systems and Data Science (ICSSD). IEEE (2019)
Hafsa, M., Jemili, F.: Comparative Study between Big Data Analysis Techniques in Intrusion Detection, p. 1. Big Data and Cognitive Computing
Belouch, M., El Hadaj, S., Idhammad, M.: Performance evaluation of intrusion detection based on machine learning using Apache Spark. Procedia Comput. Sci. 1–6 (2018)
Zhang, H., Dai, S., Li, Y., Zhang, W.: Real-time distributed-random-forest-based network intrusion detection system using Apache spark. In: 2018 IEEE 37th International Performance Computing and Communications Conference (IPCCC), pp. 1–7. IEEE (2018)
Saravanan, S., et al.: Performance evaluation of classification algorithms in the design of Apache Spark based intrusion detection system. In: 2020 5th International Conference on Communication and Electronics Systems (ICCES), pp. 443–447. IEEE (2020)
Karatas, G., Demir, O., Sahingoz, O.K.: Increasing the Performance of Machine Learning-Based IDSs on an Imbalanced and Up-to-Date Dataset, pp. 32150–32162. IEEE Access (2020)
Ferrag, M.A., Maglaras, L., Janicke, H., Smith, R.: Deep learning techniques for cyber security intrusion detection: a detailed analysis. In: 6th International Symposium for ICS & SCADA Cyber Security Research, pp. 126–136 (2019)
Lypa, B., Iver, O., Kifer, V.: Application of machine learning methods for network intrusion detection system (2019)
Kim, J., Shin, Y., Choi, E., et al.: An intrusion detection model based on a convolutional neural network. J. Multimed. Inf. Syst. 165–172 (2019)
IDS 2018 | Datasets. unb.ca/cic/datasets/ids-2018. [Online] (2018). https://www.unb.ca/cic/datasets/ids-2018.html
Foukia, N., Hulaas, J.G., Harms, J.: Intrusion Detection with Mobile Agents (2001)
Achbarou, O., El Kiram, M.A., Bourkoukou, O., Elbouanani, S.: A new distributed intrusion detection system based on multi-agent system for cloud environment. Int. J. Commun. Netw. Inf. Secur. (Kohat University of Science and Technology (KUST)) 526 (2018)
Anusha, K., Usha Rani, K.: Performance evaluation of Spark SQL for batch processing. In: Emerging Research in Data Engineering Systems and Computer Communications, pp. 145–153. Springer (2020)
Apache Sparkâ„¢. [Online] (2020). https://databricks.com/spark/about
Salloum, S., Dautov, R., Chen, X., Peng, P.X., Huang, J.Z.: Big data analytics on Apache Spark. Int. J. Data Sci. Anal. (Springer) 145–164 (2016)
Morfino, V., Rampone, S.: Towards Near-Real-Time Intrusion Detection for IoT Devices using Supervised Learning and Apache Spark, p. 444. Electronics. Multidisciplinary Digital Publishing Institute (2020)
Satapathy, S.C., Bhateja, V., Das, S.: Smart intelligent computing and applications. In: Proceedings of the Second International Conference on SCI (2018)
How to Overcome the Limitations of RDD in Apache Spark? [Online]. https://data-flair.training/blogs/apache-spark-rdd-limitations/
What-microsoft-azure-is-and-why-it-matters. [Online] (2018). https://ccbtechnology.com/what-microsoft-azure-is-and-why-it-matters/
Rachburee, N., Punlumjeak, W.: Big data analytics: feature selection and machine learning for intrusion detection on microsoft azure platform. J. Telecommun. Electron. Comput. Eng. (JTEC) 107–111 (2017)
Blob storage. [Online]. https://azure.microsoft.com/en-us/services/storage/blobs/
Quinto, B.: XGBoost, Covers and LightGBM, Spark NLP. In: Next-Generation Machine Learning with Spark. Springer
Quinto, B.: Introduction to Spark and Spark MLlib. In: Next-Generation Machine Learning with Spark, pp. 29–96. Springer (2020)
Chourasiya, R., Patel, V., Shrivastava, A.: Classification of cyber attack using machine learning technique at microsoft azure cloud. Int. Res. J. Eng. Appl. Sci. (2018)
Dahiya, P., Srivastava, D.K.: Network intrusion detection in big dataset using Spark. Procedia Comp. Sci. (Elsevier) 253–262 (2018)
Khan, M.A., Kim, J.: Toward Developing Efficient Conv-AE-Based Intrusion Detection System Using Heterogeneous Dataset, p. 1771. Electronics. Multidisciplinary Digital Publishing Institute (2020)
Ranjana, P., et al.: Anomaly detection of DDOS attacks using Hadoop. In: Emerging Research in Computing, Information, Communication and Applications, pp. 543–552. Springer (2019)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Tesnim, Y., Farah, J. (2021). A Multi-Agent-Based System for Intrusion Detection. In: Jezic, G., Chen-Burger, J., Kusek, M., Sperka, R., Howlett, R.J., Jain, L.C. (eds) Agents and Multi-Agent Systems: Technologies and Applications 2021. Smart Innovation, Systems and Technologies, vol 241. Springer, Singapore. https://doi.org/10.1007/978-981-16-2994-5_15
Download citation
DOI: https://doi.org/10.1007/978-981-16-2994-5_15
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-2993-8
Online ISBN: 978-981-16-2994-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)