Skip to main content
SpringerLink
Log in

Local Intrinsic Dimensionality of IoT Networks for Unsupervised Intrusion Detection

  • Conference paper
  • First Online:
Book cover Data and Applications Security and Privacy XXXVI (DBSec 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13383))

Included in the following conference series:

Abstract

The Internet of Things (IoT) is revolutionizing society by connecting people, devices, and environments seamlessly and providing enhanced user experience and functionalities. Security and privacy issues remain mostly ignored. Attackers can compromise devices, inject spurious packets into an IoT network, and cause severe damage. Machine learning-based Network Intrusion Detection Systems (NIDS) are often designed to detect such attacks. Most algorithms use labeled data for training the classifiers, which is difficult to obtain in a real-world setting.

In this work, we propose a novel unsupervised machine learning approach that uses properties of the IoT dataset for anomaly detection. Specifically, we propose the use of Local Intrinsic Dimensionality (LID), a theoretical complexity measurement that assesses the local manifold surrounding a point. We use LID to evaluate three modern IoT network datasets empirically, showing that for network data generated using IoT methodologies, the LID estimates of benign network packets fit into low LID estimations. Further, we find that malicious examples exhibit higher LID estimates. We use this finding to propose a new unsupervised anomaly detection algorithm, the Weighted Hamming Distance LID Estimator, which incorporates an entropy weighted Hamming distance into the LID Maximum Likelihood Estimator algorithm. We show that our proposed approach performs better on IoT network datasets than the Autoencoder, KNN, and Isolation Forests. We test the algorithm on ToN IoT, NetFlow Bot-IoT (NF Bot-IoT), and Aposemat IoT-23 (IoT-23) datasets, using leave-one-out validation to compare results.

This work was supported in part by funding from NSF under Award Number CNS 1822118, NIST, ARL, Statnett, AMI, Cyber Risk Research, NewPush, and State of Colorado Cybersecurity Center.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://paperswithcode.com/dataset/unsw-nb15.

References

  1. Ahmad, R., Alsmadi, I.: Machine learning approaches to IoT security: a systematic literature review. Internet Things. 14, 100365 (2021). https://doi.org/10.1016/j.iot.2021.100365, https://www.sciencedirect.com/science/article/pii/S2542660521000093

  2. Alaba, F.A., Othman, M., Hashem, I.A.T., Alotaibi, F.: Internet of Things security: a survey. J. Netw. Comput. App. 88, 10–28 (2017). https://doi.org/10.1016/j.jnca.2017.04.002, https://www.sciencedirect.com/science/article/pii/S1084804517301455

  3. Alsaedi, A., Moustafa, N., Tari, Z., Mahmood, A., Anwar, A.: TON_IoT telemetry dataset: a new generation dataset of IoT and IIoT for data-driven intrusion detection systems. IEEE Access 8, 165130–165150 (2020). https://doi.org/10.1109/ACCESS.2020.3022862

    Article  Google Scholar 

  4. Amsaleg, L., et al.: Estimating local intrinsic dimensionality. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 29–38. KDD 2015, Association for Computing Machinery (2015). https://doi.org/10.1145/2783258.2783405

  5. Andrea, I., Chrysostomou, C., Hadjichristofi, G.: Internet of things: security vulnerabilities and challenges. In: IEEE Symposium on Computers and Communication (ISCC), pp. 180–187 (2015). https://doi.org/10.1109/ISCC.2015.7405513

  6. Ansuini, A., Laio, A., Macke, J.H., Zoccolan, D.: Intrinsic dimension of data representations in deep neural networks. arXiv:1905.12784 [cs, stat] (2019)

  7. Azmoodeh, A., Dehghantanha, A., Choo, K.K.R.: Robust malware detection for internet of (battlefield) things devices using deep eigenspace learning. IEEE Trans. Sustain. Comput. 4(1), 88–95 (2019). https://doi.org/10.1109/TSUSC.2018.2809665

    Article  Google Scholar 

  8. Bernal, D.: 3 - Analytical techniques for damage detection and localization for assessing and monitoring civil infrastructures. In: Wang, M.L., Lynch, J.P., Sohn, H. (eds.) Sensor Technologies for Civil Infrastructures, vol. 56, pp. 67–92. Woodhead Publishing (2014). https://doi.org/10.1533/9781782422433.1.67, https://www.sciencedirect.com/science/article/pii/B978178242242650003X

  9. Bezawada, B., Bachani, M., Peterson, J., Shirazi, H., Ray, I., Ray, I.: IoTSense: behavioral fingerprinting of IoT devices. arXiv:1804.03852 [cs] (2018)

  10. Bhatia, S., Jain, A., Li, P., Kumar, R., Hooi, B.: MSTREAM: fast anomaly detection in multi-aspect streams.In: Proceedings of the Web Conference 2021, pp. 3371–3382 (2021). https://doi.org/10.1145/3442381.3450023, http://arxiv.org/abs/2009.08451

  11. Choudhary, D.: Security challenges and countermeasures for the heterogeneity of IoT applications. J. Autonom. Intell. 1, 16 (2019). https://doi.org/10.32629/jai.v1i2.25, http://en.front-sci.com/index.php/JAI/article/view/25

  12. Claise, B.: Cisco Systems NetFlow Services Export Version 9. Request for Comments RFC 3954, Internet Engineering Task Force (2004). https://doi.org/10.17487/RFC3954, https://datatracker.ietf.org/doc/rfc3954

  13. Conti, M., Dehghantanha, A., Franke, K., Watson, S.: Internet of Things security and forensics: challenges and opportunities. Future Gener. Comput. Syst. 78, 544–546 (2018). https://doi.org/10.1016/j.future.2017.07.060, http://arxiv.org/abs/1807.10438

  14. Elrawy, M.F., Awad, A.I., Hamed, H.F.A.: Intrusion detection systems for IoT-based smart environments: a survey. J. Cloud Comput. 7(1), 21 (2018). https://doi.org/10.1186/s13677-018-0123-6

  15. Garcia, S., Parmisano, A., Erquiaga, M.J.: IoT-23 dataset: a labeled dataset of malware and benign IoT traffic. (version 1.0.0) [data set] zenodo. https://www.stratosphereips.org/datasets-iot23

  16. Gorbett, M., Blanchard, N.: Utilizing network properties to detect erroneous inputs. arXiv:2002.12520 [cs] (2020)

  17. Gorbett, M., Shirazi, H., Ray, I.: The intrinsic dimensionality of IoT networks. In: Proceedings of the 2022 ACM Symposium on Access Control Models and Technologies (SACMAT) (2022)

    Google Scholar 

  18. HaddadPajouh, H., Dehghantanha, A., Khayami, R., Choo, K.K.R.: A deep Recurrent Neural Network based approach for Internet of Things malware threat hunting. Future Gener. Comput. Syst. 85, 1-9 (2018). https://doi.org/10.1016/j.future.2018.03.007, https://www.sciencedirect.com/science/article/pii/S0167739X1732486X

  19. Haefner, K., Ray, I.: ComplexIoT: Behavior-based trust for IoT networks. In: 2019 First IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA), pp. 56–65 (2019). https://doi.org/10.1109/TPS-ISA48467.2019.00016

  20. Hendrycks, D., Gimpel, K.: A Baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv:1610.02136 [cs] (2018)

  21. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (Jul 2006). https://doi.org/10.1126/science.1127647, https://www.science.org/doi/10.1126/science.1127647

  22. Kasongo, S.M., Sun, Y.: Performance analysis of intrusion detection systems using a feature selection method on the UNSW-NB15 dataset. J. Big Data 7, 105 (2020). https://doi.org/10.1186/s40537-020-00379-6

  23. Kollolu, R.: A Review on wide variety and heterogeneity of IoT platforms. SSRN Scholarly Paper ID 3912454, Social Science Research Network, Rochester, NY (2020). https://doi.org/10.2139/ssrn.3912454, https://papers.ssrn.com/abstract=3912454

  24. Koroniotis, N., Moustafa, N., Sitnikova, E., Slay, J.: towards developing network forensic mechanism for botnet activities in the IoT based on machine learning techniques. In: Mobile Networks and Management. pp. 30–44. Springer International Publishing, Cham (2018). https://doi.org/10.1007/978-3-319-90775-83

  25. Koroniotis, N., Moustafa, N., Sitnikova, E., Turnbull, B.: Towards the development of realistic botnet dataset in the Internet of Things for network forensic analytics: Bot-IoT dataset. Future Gener. Comput. Syst. 100, 779–796 (2019). https://doi.org/10.1016/j.future.2019.05.041, https://www.sciencedirect.com/science/article/pii/S0167739X18327687

  26. Kozik, R., Pawlicki, M., Choraś, M.: A new method of hybrid time window embedding with transformer-based traffic data classification in IoT-networked environment. Pattern Anal. Appl. 24(4), 1441–1449 (2021). https://doi.org/10.1007/s10044-021-00980-2, https://www.mendeley.com/catalogue/92cc3e51-9dc9-3c8e-9e05-aeea7382b93c/

  27. Levina, E., Bickel, P.J.: Maximum Likelihood estimation of intrinsic dimension. In: Proceedings of the 17th International Conference on Neural Information Processing Systems, pp. 777–784. NIPS 2004, MIT Press, Cambridge, MA, USA (2004)

    Google Scholar 

  28. Ma, X.,et al.: Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality. arXiv:1801.02613 [cs] (2018)

  29. Meftah, S., Rachidi, T., Assem, N.: Network based intrusion detection using the UNSW-NB15 dataset. Int. J. Comput. Digital Syst. 8, 478–487 (2019). https://doi.org/10.12785/ijcds/080505, https://journal.uob.edu.bh:443/handle/123456789/3580

  30. Mohsin, M., Anwar, Z., Husari, G., Al-Shaer, E., Rahman, M.A.: IoTSAT: a formal framework for security analysis of the internet of things (IoT). In: 2016 IEEE Conference on Communications and Network Security (CNS), pp. 180–188 (2016). https://doi.org/10.1109/CNS.2016.7860484

  31. Morningstar, W., Ham, C., Gallagher, A., Lakshminarayanan, B., Alemi, A., Dillon, J.: Density of states estimation for out of distribution detection. In: Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, pp. 3232–3240. PMLR (2021). https://proceedings.mlr.press/v130/morningstar21a.html

  32. Moustafa, N.: A new distributed architecture for evaluating AI-based security systems at the edge: network TON_iot datasets. Sustain. Cities Soc. 72, 102994 (2021). https://doi.org/10.1016/j.scs.2021.102994, https://www.sciencedirect.com/science/article/pii/S2210670721002808

  33. Moustafa, N., Slay, J.: UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In: 2015 Military Communications and Information Systems Conference (MilCIS), pp. 1–6 (2015). https://doi.org/10.1109/MilCIS.2015.7348942

  34. Nalisnick, E., Matsukawa, A., Teh, Y.W., Gorur, D., Lakshminarayanan, B.: Do Deep Generative Models Know What They Don’t Know? (2018). https://openreview.net/forum?id=H1xwNhCcYm

  35. Tapping AI for Intrusion Detection Systems, October 2021. https://www.iotworldtoday.com/2021/10/18/tapping-ai-for-intrusion-detection-systems/

  36. Pope, P., Zhu, C., Abdelkader, A., Goldblum, M., Goldstein, T.: The Intrinsic Dimension of Images and Its Impact on Learning (2020). https://openreview.net/forum?id=XJk19XzGq2J

  37. Rashma, B.M., Macherla, S., Jaiswal, A., Poornima, G.: Handling heterogeneity in an IoT infrastructure. In: Patnaik, S., Yang, X.-S., Sethi, I.K. (eds.) Advances in Machine Learning and Computational Intelligence. AIS, pp. 635–643. Springer, Singapore (2021). https://doi.org/10.1007/978-981-15-5243-4_60

    Chapter  Google Scholar 

  38. Rezvy, S., Luo, Y., Petridis, M., Lasebae, A., Zebin, T.: An efficient deep learning model for intrusion classification and prediction in 5G and IoT networks. In: 2019 53rd Annual Conference on Information Sciences and Systems (CISS), pp. 1–6 (2019). https://doi.org/10.1109/CISS.2019.8693059

  39. Rizvi, S., Orr, R., Cox, A., Ashokkumar, P., Rizvi, M.R.: Identifying the attack surface for IoT network. Internet Things. 9, 100162 (2020). https://doi.org/10.1016/j.iot.2020.100162, https://www.sciencedirect.com/science/article/pii/S2542660520300056

  40. Roy, B., Cheung, H.: A Deep Learning Approach for Intrusion Detection in Internet of Things using Bi-Directional Long Short-Term Memory Recurrent Neural Network, pp. 1–6 (2018). https://doi.org/10.1109/ATNAC.2018.8615294. ISSN: 2474-154X

  41. Sahu, A.K., Sharma, S., Tanveer, M., Raja, R.: Internet of Things attack detection using hybrid Deep Learning Model. Comput. Commun. 176, 146–154 (2021). https://doi.org/10.1016/j.comcom.2021.05.024, https://www.sciencedirect.com/science/article/pii/S0140366421002164

  42. Sarhan, M., Layeghy, S., Moustafa, N., Portmann, M.: NetFlow datasets for machine learning-based network intrusion detection systems. In: Deze, Z., Huang, H., Hou, R., Rho, S., Chilamkurti, N. (eds.) BDTA/WiCON -2020. LNICST, vol. 371, pp. 117–135. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72802-1_9

    Chapter  Google Scholar 

  43. Serrà, J., Álvarez, D., Gómez, V., Slizovskaia, O., Núñez, J.F., Luque, J.: Input Complexity and Out-of-distribution Detection with Likelihood-based Generative Models (2019). https://openreview.net/forum?id=SyxIWpVYvr

  44. Shone, N., Ngoc, T.N., Phai, V.D., Shi, Q.: A deep learning approach to network intrusion detection. IEEE Trans. Emerg. Topics Comput. Intell. 2, 41–50 (2018). https://doi.org/10.1109/TETCI.2017.2772792

    Article  Google Scholar 

  45. Singh, P., Jaykumar, J., Pankaj, A., Mitra, R.: Edge-Detect: Edge-centric Network Intrusion Detection using Deep Neural Network. arXiv:2102.01873 [cs], February 2021

  46. Stolz, B.J., Tanner, J., Harrington, H.A., Nanda, V.: Geometric anomaly detection in data. Proc. Natl. Acad. Sci. 117(33), 19664–19669 (2020). https://doi.org/10.1073/pnas.2001741117, https://www.pnas.org/content/117/33/19664

  47. Vasudevan, A., Harshini, E., Selvakumar, S.: SSENet-2011: a network intrusion detection system dataset and its comparison with KDD CUP 99 dataset. In: 2011 Second Asian Himalayas International Conference on Internet (AH-ICI), pp. 1–5 (2011). https://doi.org/10.1109/AHICI.2011.6113948

  48. Wang, Q., Erfani, S.M., Leckie, C., Houle, M.E.: A Dimensionality-Driven Approach for Unsupervised Out-of-distribution Detection, p. 9 (2021)

    Google Scholar 

  49. Zhao, K., Ge, L.: A Survey on the Internet of Things Security, pp. 663–667 (2013). DOI: https://doi.org/10.1109/CIS.2013.145

  50. Zhou, S., Tordesillas, A., Pouragha, M., Bailey, J., Bondell, H.: On local intrinsic dimensionality of deformation in complex materials. Sci. Rep. 11(1), 10216 (2021). https://doi.org/10.1038/s41598-021-89328-8, https://www.nature.com/articles/s41598-021-89328-8

Download references

Author information

Authors and Affiliations

  1. Colorado State University, Fort Collins, CO, 80523, USA

    Matt Gorbett, Hossein Shirazi & Indrakshi Ray

Authors
  1. Matt Gorbett
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hossein Shirazi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Indrakshi Ray
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matt Gorbett .

Editor information

Editors and Affiliations

  1. Department of Computer Science and Engineering, Indian Institute of Technology Kharagpur, Kharagpur, West Bengal, India

    Shamik Sural

  2. Department of Information Systems and Analytics, Santa Clara University, Santa Clara, CA, USA

    Haibing Lu

Rights and permissions

Reprints and permissions

Copyright information

© 2022 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gorbett, M., Shirazi, H., Ray, I. (2022). Local Intrinsic Dimensionality of IoT Networks for Unsupervised Intrusion Detection. In: Sural, S., Lu, H. (eds) Data and Applications Security and Privacy XXXVI. DBSec 2022. Lecture Notes in Computer Science, vol 13383. Springer, Cham. https://doi.org/10.1007/978-3-031-10684-2_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-10684-2_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-10683-5

  • Online ISBN: 978-3-031-10684-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation