Skip to main content
SpringerLink
Log in

Next Generation Automated Reservoir Computing for Cyber Defense

  • Conference paper
  • First Online:
Artificial Intelligence Applications and Innovations (AIAI 2023)

Part of the book series: IFIP Advances in Information and Communication Technology ((IFIPAICT,volume 676))

Abstract

It is a fact, that important information related to systems’ behavior and dynamics, can be revealed as time passes. Observing changes over time, can often lead to the detection of patterns and trends that might not be immediately apparent from a single system’s snapshot. Additionally, the concept of time can be essential in understanding cause-and-effect relationships. Observing how changes in one variable over time affect changes in another, can gain insights into the causal relationships between different system components. Time series analysis of data that change over time, can be a powerful tool for understanding complex systems in the field of cyber security. Reservoir Computing (RC) is a Machine Learning technique, using a fixed and randomly generated high-dimensional dynamic system, called a Reservoir, to transform and classify input data. The reservoir acts as a nonlinear and temporal filter of the input data, which is then readout by a linear output layer. Continuous-Time Reservoir Computing (CTRC) is a type of recurrent neural network, aiming to model the continuous-time dynamics of the network’s neurons. It is particularly useful for applications where time is critical and it can provide insights into the underlying system’s dynamics. This paper proposes a next-generation CTRC for cyber defense, where the reservoir neurons are modeled as continuous-time dynamical systems. This means that their behavior is described by a system of differential equations that change over time. In order to model the drift phenomenon, identify the abnormal changes in the data, and adaptively stabilize the learning system. The CTRC parameters are optimized using the Reinforcement Learning (RL) method. The proposed system, as proved experimentally, has several advantages over discrete systems, including the ability to handle signals with high sampling rates and to effectively capture real cyber security signals’ continuous nature.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bala, A., Ismail, I., Ibrahim, R., Sait, S.M.: Applications of metaheuristics in reservoir computing techniques: a review. IEEE Access 6, 58012–58029 (2018). https://doi.org/10.1109/ACCESS.2018.2873770

    Article  Google Scholar 

  2. Demertzis, K., Iliadis, L., Pimenidis, E.: Geo-AI to aid disaster response by memory-augmented deep reservoir computing. Integr. Comput.-Aided Eng. 28(4), 383–398 (2021). https://doi.org/10.3233/ICA-210657

    Article  Google Scholar 

  3. Freiberger, M., Katumba, A., Bienstman, P., Dambre, J.: Training passive photonic reservoirs with integrated optical readout. IEEE Trans. Neural Netw. Learn. Syst. 30(7), 1943–1953 (2019). https://doi.org/10.1109/TNNLS.2018.2874571

    Article  Google Scholar 

  4. Li, S., Pachnicke, S.: Photonic reservoir computing in optical transmission systems. In: 2020 IEEE Photonics Society Summer Topicals Meeting Series (SUM), pp. 1–2 (2020). https://doi.org/10.1109/SUM48678.2020.9161045

  5. Gauthier, D.J., Bollt, E., Griffith, A., Barbosa, W.A.S.: Next generation reservoir computing. Nat. Commun. 12(1), Art. no. 1 (2021). https://doi.org/10.1038/s41467-021-25801-2

  6. Demertzis, K., Kikiras, P., Tziritas, N., Sanchez, S.L., Iliadis, L.: The next generation cognitive security operations center: network flow forensics using cybersecurity intelligence. Big Data Cogn. Comput. 2(4), Art. no. 4 (2018). https://doi.org/10.3390/bdcc2040035

  7. Hart, A.: Generalised Synchronisation for Continuous Time Reservoir Computers. Rochester, NY (2021). https://doi.org/10.2139/ssrn.3987856

  8. Smith, L.M., Kim, J.Z., Lu, Z., Bassett, D.S.: Learning continuous chaotic attractors with a reservoir computer. Chaos Interdiscip. J. Nonlinear Sci. 32(1), 011101 (2022). https://doi.org/10.1063/5.0075572

    Article  Google Scholar 

  9. Abu, U.A., Folly, K.A., Jayawardene, I., Venayagamoorthy, G.K.: Echo state network (ESN) based generator speed prediction of wide area signals in a multimachine power system. In: 2020 International SAUPEC/RobMech/PRASA Conference, pp. 1–5 (2020). https://doi.org/10.1109/SAUPEC/RobMech/PRASA48453.2020.9041236

  10. Duport, F., Smerieri, A., Akrout, A., Haelterman, M., Massar, S.: Fully analogue photonic reservoir computer. Sci. Rep. 6(1), Art. no. 1 (2016). https://doi.org/10.1038/srep22381

  11. Manjunath, G.: An echo state network imparts a curve fitting. IEEE Trans. Neural Netw. Learn. Syst. 33(6), 2596–2604 (2022). https://doi.org/10.1109/TNNLS.2021.3099091

    Article  MathSciNet  Google Scholar 

  12. Wang, Z., Yao, X., Huang, Z., Liu, L.: Deep echo state network with multiple adaptive reservoirs for time series prediction. IEEE Trans. Cogn. Dev. Syst. 13(3), 693–704 (2021). https://doi.org/10.1109/TCDS.2021.3062177

    Article  Google Scholar 

  13. Whiteaker, B., Gerstoft, P.: Memory in echo state networks and the controllability matrix rank. In: ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3948–3952 (2022). https://doi.org/10.1109/ICASSP43922.2022.9746766

  14. Shao, Y., Yao, X., Wang, G., Cao, S.: A new improved echo state network with multiple output layers for time series prediction. In: 2021 6th International Conference on Robotics and Automation Engineering (ICRAE), pp. 7–11 (2021). https://doi.org/10.1109/ICRAE53653.2021.9657812

  15. Li, X., Bi, F., Yang, X., Bi, X.: An echo state network with improved topology for time series prediction. IEEE Sens. J. 22(6), 5869–5878 (2022). https://doi.org/10.1109/JSEN.2022.3148742

    Article  Google Scholar 

  16. Kidger, P.: On Neural Differential Equations. arXiv (2022). https://doi.org/10.48550/arXiv.2202.02435

  17. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019). https://doi.org/10.1016/j.jcp.2018.10.045

  18. Demertzis, K., Iliadis, L.: Adaptive elitist differential evolution extreme learning machines on big data: intelligent recognition of invasive species. In: Angelov, P., Manolopoulos, Y., Iliadis, L., Roy, A., Vellasco, M. (eds.) INNS 2016. AISC, vol. 529, pp. 333–345. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-47898-2_34

    Chapter  Google Scholar 

  19. Bai, Z., Pang, H., Liu, M., Wang, M.: An improved Q-Learning algorithm and its application to the optimized path planning for unmanned ground robot with obstacle avoidance. In: 2022 6th CAA International Conference on Vehicular Control and Intelligence (CVCI), pp. 1–6 (2022). https://doi.org/10.1109/CVCI56766.2022.9964859

  20. Huang, D., Zhu, H., Lin, X., Wang, L.: Application of massive parallel computation based Q-learning in system control. In: 2022 5th International Conference on Pattern Recognition and Artificial Intelligence (PRAI), pp. 1–5 (2022). https://doi.org/10.1109/PRAI55851.2022.9904213

  21. Yin, Z., Cao, W., Song, T., Yang, X., Zhang, T.: Reinforcement learning path planning based on step batch Q-learning algorithm. In: 2022 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA), pp. 630–633 (2022). https://doi.org/10.1109/ICAICA54878.2022.9844553

  22. Chouiekh, C., Yahyaouy, A., Aarab, A., Sabri, A.: Road traffic: deep q-learning agent control traffic lights in the intersection. In: 2022 International Conference on Intelligent Systems and Computer Vision (ISCV), pp. 1–5 (2022). https://doi.org/10.1109/ISCV54655.2022.9806135

  23. InfluxDB Times Series Data Platform. InfluxData (2022). https://www.influxdata.com/home/. Accessed 28 Feb 2023

  24. Industrial IoT (IIoT) solutions for smart industries – Factry. Factry - Open Manufacturing Intelligence. https://www.factry.io/. Accessed 28 Feb 2023

  25. Nguyen, Q.-D., Dhouib, S., Chanet, J.-P., Bellot, P.: Towards a web-of-things approach for OPC UA field device discovery in the industrial IoT. In: 2022 IEEE 18th International Conference on Factory Communication Systems (WFCS), pp. 1–4 (2022). https://doi.org/10.1109/WFCS53837.2022.9779181

  26. Demertzis, K., Iliadis, L.S., Anezakis, V.-D.: An innovative soft computing system for smart energy grids cybersecurity. Adv. Build. Energy Res. 12(1), 3–24 (2018). https://doi.org/10.1080/17512549.2017.1325401

    Article  Google Scholar 

  27. Wang, H., Wang, Y., Wan, S.: A density-based clustering algorithm for uncertain data. In: 2012 International Conference on Computer Science and Electronics Engineering, vol. 3, pp. 102–105 (2012). https://doi.org/10.1109/ICCSEE.2012.91

  28. Khan, M.M.R., Siddique, M.A.B., Arif, R.B., Oishe, M.R.: ADBSCAN: adaptive density-based spatial clustering of applications with noise for identifying clusters with varying densities. In: 2018 4th International Conference on Electrical Engineering and Information & Communication Technology (iCEEiCT), pp. 107–111 (2018). https://doi.org/10.1109/CEEICT.2018.8628138

  29. Botchkarev, A.: Performance metrics (error measures) in machine learning regression, forecasting and prognostics: properties and typology. Interdiscip. J. Inf. Knowl. Manag. 14, 045–076 (2019). https://doi.org/10.28945/4184

    Article  Google Scholar 

  30. Koyejo, O.O., Natarajan, N., Ravikumar, P.K., Dhillon, I.S.: Consistent binary classification with generalized performance metrics. In: Advances in Neural Information Processing Systems, vol. 27 (2014). https://papers.nips.cc/paper/2014/hash/30c8e1ca872524fbf7ea5c519ca397ee-Abstract.html. Accessed 24 Oct 2021

  31. Liu, Y., Zhou, Y., Wen, S., Tang, C.: A strategy on selecting performance metrics for classifier evaluation. Int. J. Mob. Comput. Multimed. Commun. IJMCMC 6(4), 20–35 (2014). https://doi.org/10.4018/IJMCMC.2014100102

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Science and Technology, Informatics Studies, Hellenic Open University, Patra, Greece

    Konstantinos Demertzis

  2. School of Engineering, Department of Civil Engineering, Faculty of Mathematics Programming and General Courses, Democritus University of Thrace, Kimmeria, Xanthi, Greece

    Lazaros Iliadis

Authors
  1. Konstantinos Demertzis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lazaros Iliadis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Konstantinos Demertzis .

Editor information

Editors and Affiliations

  1. University of Piraeus, Piraeus, Greece

    Ilias Maglogiannis

  2. Democritus University of Thrace, Xanthi, Greece

    Lazaros Iliadis

  3. University of Sunderland, Sunderland, UK

    John MacIntyre

  4. University of Leon, León, Spain

    Manuel Dominguez

Rights and permissions

Reprints and permissions

Copyright information

© 2023 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Demertzis, K., Iliadis, L. (2023). Next Generation Automated Reservoir Computing for Cyber Defense. In: Maglogiannis, I., Iliadis, L., MacIntyre, J., Dominguez, M. (eds) Artificial Intelligence Applications and Innovations. AIAI 2023. IFIP Advances in Information and Communication Technology, vol 676. Springer, Cham. https://doi.org/10.1007/978-3-031-34107-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-34107-6_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-34106-9

  • Online ISBN: 978-3-031-34107-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation