Skip to main content

Advertisement

SpringerLink
Log in

A novel energy-based online sequential extreme learning machine to detect anomalies over real-time data streams

  • Special issue on Advances of Neural Computing phasing challenges in the era of 4th industrial revolution
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Data flow learning algorithms must be very efficient in learning and predicting sequences. The model that monitors a sequence of data or events can predict the sequel and can act in such a way that it optimally achieves the desired result. Security and digital risk tracking systems are receiving a constant and unlimited input of observations. These data flows are characterized by high variability, as their properties can change drastically and unpredictably over time. Each incoming example can only be processed once, or it must be summarized with a small memory imprint. This research paper proposes the development of an intelligent system, for real-time detection of data flow anomalies related to information systems’ security. Specifically, it describes the implementation of an efficient and high-precision energy-based Online Sequential Extreme Learning Machine (e-b OSELM) that is proposed for the first time in the literature. It is an intelligent model that can detect data dependencies, by applying a measure of compatibility (scalable energy) to each configuration of its variables. It assigns low energy to the correct values and higher energy to the divergent (abnormal) ones. The innovative combination of energy models and ELMs offers high learning speed, ease of execution, minimum human involvement and minimum computational power and resources for anomaly detection and identification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Sample C, Schaffer K (2013) An overview of anomaly detection. IT Prof 15(1):8–11

    Article  Google Scholar 

  2. Callegari C, Giordano S, Pagano M (2017) Anomaly detection: an overview of selected methods. In: 2017 international multi-conference on engineering, computer and information sciences (SIBIRCON), Novosibirsk, 2017, pp 52–57

  3. Bhuyan MH, Bhattacharyya DK, Kalita JK (2014) Network anomaly detection: methods, systems and tools. In: IEEE communications surveys & tutorials, vol. 16, no. 1, pp 303–336, First Quarter

  4. Rana AI, Estrada G, Solé M, Muntés V (2016) Anomaly detection guidelines for data streams in big data. In: 2016 3rd international conference on soft computing & machine intelligence (ISCMI), Dubai, 2016, pp 94–98

  5. Rettig L, Khayati M, Cudré-Mauroux P, Piórkowski M (2015) Online anomaly detection over Big Data streams. In: 2015 IEEE international conference on big data (Big Data), Santa Clara, CA, 2015, pp 1113–1122

  6. Guler ER, Ozdemir S (2018) Applications of stream data mining on the internet of things: a survey. In: 2018 International congress on big data, deep learning and fighting cyber terrorism (IBIGDELFT), ANKARA, Turkey, 2018, pp 51–55

  7. Robins A (1993) Catastrophic forgetting in neural networks: the role of rehearsal mechanisms. In: Proceedings 1993 the first new zealand international two-stream conference on artificial neural networks and expert systems, Dunedin, New Zealand, pp 65–68

  8. Tellis VM, D'Souza DJ (2018) Detecting anomalies in data stream using efficient techniques: a review. In: 2018 International conference on control, power, communication and computing technologies (ICCPCCT), Kannur, 2018, pp 296–298

  9. Barnett V, Lewis T (1994) Outliers in statistical data. Wiley, New York

    MATH  Google Scholar 

  10. Rousseeuw LAS, MMKDP (2016) A survey on outlier detection technique in streaming data using data clustering approach. Int J Eng Comput Sci 5:1

  11. Ramaswamy S, Rastogi R, Kyuseok S (2000) Efficient algorithms for mining outliers from large data Sets. In: Proceedings of the ACMSIDMOD international conference on management of data

  12. Chugh N, Chugh M, Agarwal A (2014) Outlier detection in streaming data a research perspective. In: International conference on parallel, distributed and grid computing, IEEE

  13. Pokrajac D, Lazarevic DA, Latecki LJ (2007) Incremental local outlier detection for data streams. In: CIDM 2007. IEEE symposium on computational intelligence and data mining, 2007, pp 504–515. IEEE

  14. Karimian SH, Kelarestaghi M, Hashemi S (2012) IIncLOF: Improved incremental local outlier detection for data streams. In: 16th CSI International Symposium on Artificial Intelligence and Signal Processing (AISP 2012), (pp. 023-028). IEEE

  15. Thakran Y, Toshniwal D (2012) Unsupervised outlier detection in streaming data using weighted clustering. In: 2012 12th international conference on intelligent systems design and applications (ISDA), pp 947–952. IEEE

  16. Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv 41(3):15

    Article  Google Scholar 

  17. Jakhale AR (2017) Design of anomaly packet detection framework by data mining algorithm for network flow. In: 2017 international conference on computational intelligence in data science (ICCIDS), Chennai, pp 1–6

  18. Abbady S, Ke C, Lavergne J, Chen J, Raghavan V, Benton R (2017) Online mining for association rules and collective anomalies in data streams. In: 2017 IEEE international conference on big data (big data), Boston, MA, pp 2370–2379

  19. Miao X, Liu Y, Zhao H, Li C (2019) Distributed online one-class support vector machine for anomaly detection over networks. IEEE Trans Cybernet 49(4):1475–1488

    Article  Google Scholar 

  20. Mozaffari M, Yilmaz Y (2019) Online anomaly detection in multivariate settings. In: 2019 IEEE 29th international workshop on machine learning for signal processing (MLSP), Pittsburgh, PA, USA, pp 1–6

  21. Scardapane S, Comminiello D, Scarpiniti M, Uncini A (2015) Online Sequential Extreme Learning Machine With Kernels. IEEE Trans Neural Netw Learn Syst 26(9):2214–2220

    Article  MathSciNet  Google Scholar 

  22. Li L, Zeng J, Jiao L, Liang P, Liu F, Yang S (2020) Online active extreme learning machine with discrepancy sampling for PolSAR classification. IEEE Trans Geosci Remote Sens 58(3):2027–2041

    Article  Google Scholar 

  23. Lu X, Liu C, Huang M (2015) Online probabilistic extreme learning machine for distribution modeling of complex batch forging processes. IEEE Trans Industr Inf 11(6):1277–1286

    Article  Google Scholar 

  24. Vijendran S, Dubey R (2019) Deep online sequential extreme learning machines and its application in pneumonia detection. In: 2019 8th international conference on industrial technology and management (ICITM). Cambridge, UK, pp 311–316

  25. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1):489–501

    Article  Google Scholar 

  26. Huang G-B (2015) What are extreme learning machines? Filling the gap between Frank Rosenblatt’s Dream and John von Neumann’s Puzzle. Cognitive Comput 7:263–278

    Article  Google Scholar 

  27. Kasun LLC, Zhou H, Huang G-B, Vong CM (2013) Representational learning with extreme learning machine for big data. IEEE Intell Syst 28(6):31–34

    Google Scholar 

  28. Salakhutdinov R, Mnih A, Hinton G (2007) Restricted Boltzmann machines for collaborative filtering. In: Proceedings of the 24th international conference on machine learning-ICML '07, p 791

  29. Osadchy R, Miller M, LeCun Y (2005) Synergistic face detection and pose estimation with energy-based model. In: Advances in neural information processing systems (NIPS 2004). MIT Press

  30. Lecun Y, Chopra S, Hadsell R, Ranzato MA, Huang FJ (2006) A tutorial on energy-based learning. In: Bakir G, Hofman T, Scholkopt B, Smola A, Taskar B (eds) Predicting structured data. MIT Press, New York

    Google Scholar 

  31. Recht B, Re C (2012) Beneath the valley of the noncommutative arithmetic-geometric mean inequality: conjectures, case-studies, and consequences. arXiv preprint arXiv:1202.4184

  32. Lockhart RA, O’Reilly FJ, Stephens MA (2007) Use of the Gibbs sampler to obtain conditional tests, with applications. Biometrika 94:992–998

    Article  MathSciNet  Google Scholar 

  33. Martino L, Read J, Luengo D (2015) Independent doubly adaptive rejection metropolis sampling Within Gibbs sampling. IEEE Trans Signal Process 63(12):3123–3138

  34. Gamerman D, Lopes HF (2006) Markov Chain Monte Carlo: stochastic simulation for Bayesian inference, 2nd edn. Chapmal & Hall/CRC, London

    Book  Google Scholar 

  35. Robert C, Casella G (2011) A short history of Markov chain Monte Carlo: subjective recollections from incomplete data. Stat Sci 26:102–115

    MathSciNet  MATH  Google Scholar 

  36. Tierney L (1994) Markov chains for exploring posterior distributions. Ann Stat 22(4):1701–1728. https://doi.org/10.2307/2242477

    Article  MathSciNet  MATH  Google Scholar 

  37. Bhosale KS, Nenova M, Iliev G (2017) The distributed denial of service attacks (DDoS) prevention mechanisms on application layer. In: 2017 13th international conference on advanced technologies, systems and services in telecommunications (TELSIKS), Nis, 2017, pp 136–139

  38. Sharafaldin I, Lashkari AH, Hakak S, Ghorbani AA (2019) Developing realistic distributed denial of service (DDoS) attack dataset and taxonomy. In: IEEE 53rd international Carnahan conference on security technology, Chennai, India

  39. https://cloud.google.com/

  40. https://spark.apache.org/

  41. https://www.tensorflow.org/

  42. Lachiche N, Flach PA (2003) Improving accuracy and cost of two-class and multi-class probabilistic classifiers using ROC curves. In: T Fawcett, N Mishra (eds) Proceedings of the 20th international conference on machine learning (ICML'03), pp 416–423. AAAI Press

  43. Hand DJ, Till RJ (2001) A simple generalisation of the area under the roc curve for multiple class classification problems. Machine Learn 45:171–186

    Article  Google Scholar 

  44. Demertzis K, Iliadis L (2016) Ladon: a cyber-threat bio-inspired intelligence management system. J Appl Math Bioinform 6:45–64

    Google Scholar 

  45. Rantos K, Drosatos G, Demertzis K, Ilioudis C, Papanikolaou A, Kritsas A (2019) ADvoCATE: a consent management platform for personal data processing in the IoT using blockchain technology. In: Lanet JL, Toma C (eds) Innovative security solutions for information technology and communications. SECITC 2018 Lecture Notes in Computer Science. Springer, Cham

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by the China National Key Research and Development Program under Grant 2018YFB0803600, in part by the Natural Science Foundation of China under Grant 61801008, in part by the Scientific Research Common Program of Beijing Municipal Commission of Education under Grant KM201910005025, in part by the Chinese Postdoctoral Science Foundation under Grant 2020M670074 and in part by Defense Industrial Technology Development Program (No. JCKY2016204A102).

Author information

Authors and Affiliations

  1. Beijing Key Laboratory of Trusted Computing, Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China

    Xiaoping Wang & Shanshan Tu

  2. Beijing Electro-Mechanical Engineering Institute, Beijing, China

    Wei Zhao

  3. Institute of Information Engineering, Chinese Academy of Sciences, Beijing, 100195, China

    Chengjie Shi

Authors
  1. Xiaoping Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shanshan Tu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chengjie Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shanshan Tu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Tu, S., Zhao, W. et al. A novel energy-based online sequential extreme learning machine to detect anomalies over real-time data streams. Neural Comput & Applic 34, 823–831 (2022). https://doi.org/10.1007/s00521-021-05731-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-05731-2

Keywords

Search

Navigation