Deep Learning Clusters in the Cognitive Packet Network

Abstract

The Cognitive Packet Network (CPN) bases its routing decisions and flow control on the Random Neural Network (RNN) Reinforcement Learning algorithm; this paper proposes the addition of a Deep Learning (DL) Cluster management structure to the CPN for Quality of Service metrics (Delay Loss and Bandwidth), Cyber Security keys (User, Packet and Node) and Management decisions (QoS, Cyber and CEO). The RNN already models how neurons transmit information using positive and negative impulsive signals whereas the proposed additional Deep Learning structure emulates the way the brain learns and takes decisions; this paper presents a brain model as the combination of both learning algorithms, RNN and DL. The proposed model has been simulated under different network sizes and scenarios and it has been validated against the CPN itself without DL clusters. The simulation results are promising; the presented CPN with DL clusters as a mechanism to transmit, learn and make packet routing decisions is a step closer to emulate the way the brain transmits information, learns the environment and takes decisions.

Introduction

Our brain performs several functions at the same time; it learns about the environment from our five senses; it stores memories to preserve our identity; it takes decisions on different situations and finally; it protects itself against external treats or attacks. Our brain is formed by clusters of neurons [1] specialized in learning from different senses where information is transmitted as positive and negative spikes or impulses. It functions with two types of memories [2]; short term memory is used for fast decisions and task related actions whereas long term memory preserves our identity and security. Another brain duality consists on its two operation modes [3]; consciousness under normal activities and unconsciousness under emergency situations such as being under external attack or routine operations like storing information while sleeping.

The expansion of the connectivity provided by the Ethernet and Internet protocols has enabled new industrial, technological and social applications and services however users are increasingly under new cybersecurity threats and risks. Ericsson [4] introduces cybersecurity issues and threats within Power Communications Systems in a smart grid infrastructure where network vulnerabilities and information security domains are analyzed. Ten et al. [5] present a survey on cybersecurity of critical infrastructure; in addition they propose a SCADA framework based on four procedures: real time monitoring, anomaly detection, impact analysis and mitigation strategy. They model an attack tree analysis with an algorithm for cybersecurity evaluation that incorporates password policies and port auditing. Cruz et al. [6] present a distributed intrusion detection system for SCADA systems that includes different types of security agents tuned for each specific domain: development of network, device and process level capabilities, integration of signature and anomaly based techniques against threats and finally the adoption of a distributed multi layered design with message queues to transmit predefined events between elements. Wang et al. [7] propose a framework to facilitate the development of adversary resistant Deep Neural Networks (DNN) by inserting a data transformation module between the sample and the DNN that avoids threat samples with a minimum impact on the classification accuracy. Tuor et al. [8] present an unsupervised Deep Learning approach to detect anomalous network activity from system logs in real time where events are extracted as features and the DNN learns users’ normal behaviour or anomaly as potential malicious behaviour. Wu et al. [9] present a classification of cyber physical attacks and risks in cyber manufacturing systems with possible mitigation measures such as supervised machine learning for classification and unsupervised machine learning for anomaly detection on physical data. Kim [10] proposes a new cyber defensive computer control system architecture based on the diversification of hardware systems and unidirectional communications assuming that the detection and prevention of cyber attacks will never be complete.

Deep Learning is characterized for using a cascade of L-layers of non linear processing units for feature extraction and transformation; each successive layer uses the output from the previous layer as input. Deep Learning learns multiple layers of representations that correspond to different levels of abstractions; those levels form a hierarchy of concepts where the higher the level, the more abstract concepts are learned. Schmidhuber [24] examines Deep Learning in neural networks. Bengio et al. [25] review recent work in the area of unsupervised feature learning and Deep Learning including advances in probabilistic models. They propose a new probabilistic framework to include likelihood based probabilistic models, reconstruction based models such as auto encoder variants and geometrically based manifold learning approaches. Jiea et al. [26] propose a progressive framework to deep optimize neural networks. They combine the stability of linear methods with the ability of learning complex and abstract internal representations of Deep Learning methods. They insert a linear loss layer between the input layer and the first hidden non-linear layer of a traditional Deep Learning model. Le et al. [27] study the advantages and disadvantages of off-the shelf optimization algorithms in the context of simplification and speed up the process of pre-training the unsupervised feature learning and Deep Learning. Ngiam et al. [28] propose an application of deep networks to learn features over multiple modalities to demonstrate that cross modality feature learning performs better than single modality learning. Sutskever et al. [29] present an approach to sequence learning that makes minimal assumptions on the sequence structure using a multi-layered Long Short Term Memory (LSTM) to map the input sequence to a vector of a fixed dimensionality. Bekker et al. [30] propose an intra cluster training strategy for Deep Learning with applications to language identification where the language clusters are used to define a cost function to train a neural network.

The concept of Cognitive Packet Networks or Artificial Intelligence in network routing has also been researched. Li and Zhang [37] define the architecture of Network Artificial Intelligence (NAI) that includes key components and key protocol extension requirements for self-adjustment, self-optimization, self-recovery of the network through collection of Big data of network state and machine learning. Zhang et al. [38] propose a collaborative Internet architecture that removes the restrictions from the resource/location binding, user/network binding, and control/data binding, which are the root causes of the current Internet's issues. Qadir et al. [39] provide a vision how Artificial Intelligence can simplify network management such as cloud computing, network functions virtualization, and software-defined networking where intelligent services and cognitive networks will show network-wide intelligent behaviour to solve problems of network heterogeneity, performance, and quality of service (QoS). Quan et al. [40] investigate a new Smart Identifier NETworking (SINET) prototype and propose a customized solution that enables crowd collaborations for software defined vehicular networks through crowd sensing where network function allocations are organized with a group of components with similar function.

This paper presents the association of the most complex biological system; our brain with the most complex artificial system represented in large data networks: the Internet; the information infrastructure of the Big Data and the Web. The link between both of them is the Random Neural Network [16], [17], [18]. Data networks collect information from users and transmit it to different locations; to perform this activity, they are required to make routing decisions based on different Quality of Service metrics while storing routing tables in memory under the threat of Cyber attacks.

This paper proposes the Cognitive Packet Network (CPN) [11], [12], [13], [14], [15] with an additional Deep Learning (DL) cluster [31], [32] structure that emulates how the brain operates. The proposed model adds a layer of specialised Deep Learning management clusters that take the final routing decision; DL clusters behave as a long term memory to remember network identity: QoS metrics and Cyber keys. The CPN-RNN routing algorithm is chosen under normal or conscious operations due its fast and adaptable route learning as short memory whereas DL cluster route is selected when the network is under external cyber attacks. DL clusters take routing decisions based on the long term memory in unconsciousness operation as a safe and resilient although inefficient and inflexible routing.

The mathematical model of CPN with DL clusters is described in Section 2. The implementation of the CPN-DL is defined in Section 3. The validation of the proposed model under different QoS and Cyber scenarios in small (3 × 3, 4 × 4, 5 × 5), medium (6 × 6, 7 × 7) and large square configuration node networks (8 × 8, 9 × 9, 10 × 10) from one up to eight decision layers, respectively, is presented in Section 4. Final conclusions are presented in Section 5, and related bibliography is presented at the endof the references.