Abstract
The Internet of Things (IoT) is revolutionizing society by connecting people, devices, and environments seamlessly and providing enhanced user experience and functionalities. Security and privacy issues remain mostly ignored. Attackers can compromise devices, inject spurious packets into an IoT network, and cause severe damage. Machine learning-based Network Intrusion Detection Systems (NIDS) are often designed to detect such attacks. Most algorithms use labeled data for training the classifiers, which is difficult to obtain in a real-world setting.
In this work, we propose a novel unsupervised machine learning approach that uses properties of the IoT dataset for anomaly detection. Specifically, we propose the use of Local Intrinsic Dimensionality (LID), a theoretical complexity measurement that assesses the local manifold surrounding a point. We use LID to evaluate three modern IoT network datasets empirically, showing that for network data generated using IoT methodologies, the LID estimates of benign network packets fit into low LID estimations. Further, we find that malicious examples exhibit higher LID estimates. We use this finding to propose a new unsupervised anomaly detection algorithm, the Weighted Hamming Distance LID Estimator, which incorporates an entropy weighted Hamming distance into the LID Maximum Likelihood Estimator algorithm. We show that our proposed approach performs better on IoT network datasets than the Autoencoder, KNN, and Isolation Forests. We test the algorithm on ToN IoT, NetFlow Bot-IoT (NF Bot-IoT), and Aposemat IoT-23 (IoT-23) datasets, using leave-one-out validation to compare results.
This work was supported in part by funding from NSF under Award Number CNS 1822118, NIST, ARL, Statnett, AMI, Cyber Risk Research, NewPush, and State of Colorado Cybersecurity Center.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ahmad, R., Alsmadi, I.: Machine learning approaches to IoT security: a systematic literature review. Internet Things. 14, 100365 (2021). https://doi.org/10.1016/j.iot.2021.100365, https://www.sciencedirect.com/science/article/pii/S2542660521000093
Alaba, F.A., Othman, M., Hashem, I.A.T., Alotaibi, F.: Internet of Things security: a survey. J. Netw. Comput. App. 88, 10–28 (2017). https://doi.org/10.1016/j.jnca.2017.04.002, https://www.sciencedirect.com/science/article/pii/S1084804517301455
Alsaedi, A., Moustafa, N., Tari, Z., Mahmood, A., Anwar, A.: TON_IoT telemetry dataset: a new generation dataset of IoT and IIoT for data-driven intrusion detection systems. IEEE Access 8, 165130–165150 (2020). https://doi.org/10.1109/ACCESS.2020.3022862
Amsaleg, L., et al.: Estimating local intrinsic dimensionality. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 29–38. KDD 2015, Association for Computing Machinery (2015). https://doi.org/10.1145/2783258.2783405
Andrea, I., Chrysostomou, C., Hadjichristofi, G.: Internet of things: security vulnerabilities and challenges. In: IEEE Symposium on Computers and Communication (ISCC), pp. 180–187 (2015). https://doi.org/10.1109/ISCC.2015.7405513
Ansuini, A., Laio, A., Macke, J.H., Zoccolan, D.: Intrinsic dimension of data representations in deep neural networks. arXiv:1905.12784 [cs, stat] (2019)
Azmoodeh, A., Dehghantanha, A., Choo, K.K.R.: Robust malware detection for internet of (battlefield) things devices using deep eigenspace learning. IEEE Trans. Sustain. Comput. 4(1), 88–95 (2019). https://doi.org/10.1109/TSUSC.2018.2809665
Bernal, D.: 3 - Analytical techniques for damage detection and localization for assessing and monitoring civil infrastructures. In: Wang, M.L., Lynch, J.P., Sohn, H. (eds.) Sensor Technologies for Civil Infrastructures, vol. 56, pp. 67–92. Woodhead Publishing (2014). https://doi.org/10.1533/9781782422433.1.67, https://www.sciencedirect.com/science/article/pii/B978178242242650003X
Bezawada, B., Bachani, M., Peterson, J., Shirazi, H., Ray, I., Ray, I.: IoTSense: behavioral fingerprinting of IoT devices. arXiv:1804.03852 [cs] (2018)
Bhatia, S., Jain, A., Li, P., Kumar, R., Hooi, B.: MSTREAM: fast anomaly detection in multi-aspect streams.In: Proceedings of the Web Conference 2021, pp. 3371–3382 (2021). https://doi.org/10.1145/3442381.3450023, http://arxiv.org/abs/2009.08451
Choudhary, D.: Security challenges and countermeasures for the heterogeneity of IoT applications. J. Autonom. Intell. 1, 16 (2019). https://doi.org/10.32629/jai.v1i2.25, http://en.front-sci.com/index.php/JAI/article/view/25
Claise, B.: Cisco Systems NetFlow Services Export Version 9. Request for Comments RFC 3954, Internet Engineering Task Force (2004). https://doi.org/10.17487/RFC3954, https://datatracker.ietf.org/doc/rfc3954
Conti, M., Dehghantanha, A., Franke, K., Watson, S.: Internet of Things security and forensics: challenges and opportunities. Future Gener. Comput. Syst. 78, 544–546 (2018). https://doi.org/10.1016/j.future.2017.07.060, http://arxiv.org/abs/1807.10438
Elrawy, M.F., Awad, A.I., Hamed, H.F.A.: Intrusion detection systems for IoT-based smart environments: a survey. J. Cloud Comput. 7(1), 21 (2018). https://doi.org/10.1186/s13677-018-0123-6
Garcia, S., Parmisano, A., Erquiaga, M.J.: IoT-23 dataset: a labeled dataset of malware and benign IoT traffic. (version 1.0.0) [data set] zenodo. https://www.stratosphereips.org/datasets-iot23
Gorbett, M., Blanchard, N.: Utilizing network properties to detect erroneous inputs. arXiv:2002.12520 [cs] (2020)
Gorbett, M., Shirazi, H., Ray, I.: The intrinsic dimensionality of IoT networks. In: Proceedings of the 2022 ACM Symposium on Access Control Models and Technologies (SACMAT) (2022)
HaddadPajouh, H., Dehghantanha, A., Khayami, R., Choo, K.K.R.: A deep Recurrent Neural Network based approach for Internet of Things malware threat hunting. Future Gener. Comput. Syst. 85, 1-9 (2018). https://doi.org/10.1016/j.future.2018.03.007, https://www.sciencedirect.com/science/article/pii/S0167739X1732486X
Haefner, K., Ray, I.: ComplexIoT: Behavior-based trust for IoT networks. In: 2019 First IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA), pp. 56–65 (2019). https://doi.org/10.1109/TPS-ISA48467.2019.00016
Hendrycks, D., Gimpel, K.: A Baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv:1610.02136 [cs] (2018)
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (Jul 2006). https://doi.org/10.1126/science.1127647, https://www.science.org/doi/10.1126/science.1127647
Kasongo, S.M., Sun, Y.: Performance analysis of intrusion detection systems using a feature selection method on the UNSW-NB15 dataset. J. Big Data 7, 105 (2020). https://doi.org/10.1186/s40537-020-00379-6
Kollolu, R.: A Review on wide variety and heterogeneity of IoT platforms. SSRN Scholarly Paper ID 3912454, Social Science Research Network, Rochester, NY (2020). https://doi.org/10.2139/ssrn.3912454, https://papers.ssrn.com/abstract=3912454
Koroniotis, N., Moustafa, N., Sitnikova, E., Slay, J.: towards developing network forensic mechanism for botnet activities in the IoT based on machine learning techniques. In: Mobile Networks and Management. pp. 30–44. Springer International Publishing, Cham (2018). https://doi.org/10.1007/978-3-319-90775-83
Koroniotis, N., Moustafa, N., Sitnikova, E., Turnbull, B.: Towards the development of realistic botnet dataset in the Internet of Things for network forensic analytics: Bot-IoT dataset. Future Gener. Comput. Syst. 100, 779–796 (2019). https://doi.org/10.1016/j.future.2019.05.041, https://www.sciencedirect.com/science/article/pii/S0167739X18327687
Kozik, R., Pawlicki, M., Choraś, M.: A new method of hybrid time window embedding with transformer-based traffic data classification in IoT-networked environment. Pattern Anal. Appl. 24(4), 1441–1449 (2021). https://doi.org/10.1007/s10044-021-00980-2, https://www.mendeley.com/catalogue/92cc3e51-9dc9-3c8e-9e05-aeea7382b93c/
Levina, E., Bickel, P.J.: Maximum Likelihood estimation of intrinsic dimension. In: Proceedings of the 17th International Conference on Neural Information Processing Systems, pp. 777–784. NIPS 2004, MIT Press, Cambridge, MA, USA (2004)
Ma, X.,et al.: Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality. arXiv:1801.02613 [cs] (2018)
Meftah, S., Rachidi, T., Assem, N.: Network based intrusion detection using the UNSW-NB15 dataset. Int. J. Comput. Digital Syst. 8, 478–487 (2019). https://doi.org/10.12785/ijcds/080505, https://journal.uob.edu.bh:443/handle/123456789/3580
Mohsin, M., Anwar, Z., Husari, G., Al-Shaer, E., Rahman, M.A.: IoTSAT: a formal framework for security analysis of the internet of things (IoT). In: 2016 IEEE Conference on Communications and Network Security (CNS), pp. 180–188 (2016). https://doi.org/10.1109/CNS.2016.7860484
Morningstar, W., Ham, C., Gallagher, A., Lakshminarayanan, B., Alemi, A., Dillon, J.: Density of states estimation for out of distribution detection. In: Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, pp. 3232–3240. PMLR (2021). https://proceedings.mlr.press/v130/morningstar21a.html
Moustafa, N.: A new distributed architecture for evaluating AI-based security systems at the edge: network TON_iot datasets. Sustain. Cities Soc. 72, 102994 (2021). https://doi.org/10.1016/j.scs.2021.102994, https://www.sciencedirect.com/science/article/pii/S2210670721002808
Moustafa, N., Slay, J.: UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In: 2015 Military Communications and Information Systems Conference (MilCIS), pp. 1–6 (2015). https://doi.org/10.1109/MilCIS.2015.7348942
Nalisnick, E., Matsukawa, A., Teh, Y.W., Gorur, D., Lakshminarayanan, B.: Do Deep Generative Models Know What They Don’t Know? (2018). https://openreview.net/forum?id=H1xwNhCcYm
Tapping AI for Intrusion Detection Systems, October 2021. https://www.iotworldtoday.com/2021/10/18/tapping-ai-for-intrusion-detection-systems/
Pope, P., Zhu, C., Abdelkader, A., Goldblum, M., Goldstein, T.: The Intrinsic Dimension of Images and Its Impact on Learning (2020). https://openreview.net/forum?id=XJk19XzGq2J
Rashma, B.M., Macherla, S., Jaiswal, A., Poornima, G.: Handling heterogeneity in an IoT infrastructure. In: Patnaik, S., Yang, X.-S., Sethi, I.K. (eds.) Advances in Machine Learning and Computational Intelligence. AIS, pp. 635–643. Springer, Singapore (2021). https://doi.org/10.1007/978-981-15-5243-4_60
Rezvy, S., Luo, Y., Petridis, M., Lasebae, A., Zebin, T.: An efficient deep learning model for intrusion classification and prediction in 5G and IoT networks. In: 2019 53rd Annual Conference on Information Sciences and Systems (CISS), pp. 1–6 (2019). https://doi.org/10.1109/CISS.2019.8693059
Rizvi, S., Orr, R., Cox, A., Ashokkumar, P., Rizvi, M.R.: Identifying the attack surface for IoT network. Internet Things. 9, 100162 (2020). https://doi.org/10.1016/j.iot.2020.100162, https://www.sciencedirect.com/science/article/pii/S2542660520300056
Roy, B., Cheung, H.: A Deep Learning Approach for Intrusion Detection in Internet of Things using Bi-Directional Long Short-Term Memory Recurrent Neural Network, pp. 1–6 (2018). https://doi.org/10.1109/ATNAC.2018.8615294. ISSN: 2474-154X
Sahu, A.K., Sharma, S., Tanveer, M., Raja, R.: Internet of Things attack detection using hybrid Deep Learning Model. Comput. Commun. 176, 146–154 (2021). https://doi.org/10.1016/j.comcom.2021.05.024, https://www.sciencedirect.com/science/article/pii/S0140366421002164
Sarhan, M., Layeghy, S., Moustafa, N., Portmann, M.: NetFlow datasets for machine learning-based network intrusion detection systems. In: Deze, Z., Huang, H., Hou, R., Rho, S., Chilamkurti, N. (eds.) BDTA/WiCON -2020. LNICST, vol. 371, pp. 117–135. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72802-1_9
Serrà, J., Álvarez, D., Gómez, V., Slizovskaia, O., Núñez, J.F., Luque, J.: Input Complexity and Out-of-distribution Detection with Likelihood-based Generative Models (2019). https://openreview.net/forum?id=SyxIWpVYvr
Shone, N., Ngoc, T.N., Phai, V.D., Shi, Q.: A deep learning approach to network intrusion detection. IEEE Trans. Emerg. Topics Comput. Intell. 2, 41–50 (2018). https://doi.org/10.1109/TETCI.2017.2772792
Singh, P., Jaykumar, J., Pankaj, A., Mitra, R.: Edge-Detect: Edge-centric Network Intrusion Detection using Deep Neural Network. arXiv:2102.01873 [cs], February 2021
Stolz, B.J., Tanner, J., Harrington, H.A., Nanda, V.: Geometric anomaly detection in data. Proc. Natl. Acad. Sci. 117(33), 19664–19669 (2020). https://doi.org/10.1073/pnas.2001741117, https://www.pnas.org/content/117/33/19664
Vasudevan, A., Harshini, E., Selvakumar, S.: SSENet-2011: a network intrusion detection system dataset and its comparison with KDD CUP 99 dataset. In: 2011 Second Asian Himalayas International Conference on Internet (AH-ICI), pp. 1–5 (2011). https://doi.org/10.1109/AHICI.2011.6113948
Wang, Q., Erfani, S.M., Leckie, C., Houle, M.E.: A Dimensionality-Driven Approach for Unsupervised Out-of-distribution Detection, p. 9 (2021)
Zhao, K., Ge, L.: A Survey on the Internet of Things Security, pp. 663–667 (2013). DOI: https://doi.org/10.1109/CIS.2013.145
Zhou, S., Tordesillas, A., Pouragha, M., Bailey, J., Bondell, H.: On local intrinsic dimensionality of deformation in complex materials. Sci. Rep. 11(1), 10216 (2021). https://doi.org/10.1038/s41598-021-89328-8, https://www.nature.com/articles/s41598-021-89328-8
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 IFIP International Federation for Information Processing
About this paper
Cite this paper
Gorbett, M., Shirazi, H., Ray, I. (2022). Local Intrinsic Dimensionality of IoT Networks for Unsupervised Intrusion Detection. In: Sural, S., Lu, H. (eds) Data and Applications Security and Privacy XXXVI. DBSec 2022. Lecture Notes in Computer Science, vol 13383. Springer, Cham. https://doi.org/10.1007/978-3-031-10684-2_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-10684-2_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-10683-5
Online ISBN: 978-3-031-10684-2
eBook Packages: Computer ScienceComputer Science (R0)