A Feature Selection-based Approach for the Identification of Critical Components in Complex Technical Infrastructures: Application to the CERN Large Hadron Collider

https://doi.org/10.1016/j.ress.2020.106974Get rights and content

Highlights

  • This work proposes a method for critical components identification in CTIs.

  • The method is based on the analysis of the monitoring data.

  • The critical components identification is addressed as a feature selection problem.

  • The method combines wrapper feature selection and binary classification techniques.

  • The method is validated using data collected from the CERN Large Hadron Collider.

Abstract

Complex Technical Infrastructures (CTIs) are large-scale systems made of tens of thousands of interdependent components organized in complex hierarchical architectures. They evolve in time in a way that at one point their functional logic may be more complex than originally designed, and, therefore, traditional reliability/risk importance measures cannot be used for identifying the critical components on which the protection and recovery efforts should be primarily allocated. We propose an approach for identifying the most critical components based on the large amount of operational data collected from the CTI monitoring systems over long time periods and under different operational settings. The underlying idea is to develop binary classifiers to associate different combinations of measured signals to the CTI operating or failed state. The critical CTI components are those whose condition monitoring signals allow optimally classifying the CTI state. To identify the signals and to build the classifier, we consider a feature selection wrapper approach based on the combined use of Support Vector Machine classifiers and the Binary Differential Evolution algorithm for optimization. The approach is successfully applied to a real dataset collected from the CERN (European Centre for Nuclear Research) Large Hadron Collider, a CTI for experiments of physics.

Introduction

Complex Technical Infrastructures (CTIs) are large-scale systems of systems made by tens of thousands of mutually interconnected components performing different functions and using technologies belonging to various domains (e.g. mechanics, hydraulics, electronics, Information and Communications Technology (ICT) [7], [19], [35], [60] Due to the complexity of their topology, their geographic distribution and the differences in their technologies and functionalities, the various systems are designed and built independently, and then assembled considering only the direct physical interfaces and assuming a certain number of functional dependencies based on the theoretical operational scenarios [19], [32], [40]. During the CTI life, systems change in time (e.g. grow in size, include new components, update old components because of technology advancements, consolidations and operational needs) to continue their operation, enhance or enlarge their functions, etc. The changes that occur modify the physical interconnections among the systems, which, consequently, change the functional dependencies among their components, and the criticality of the functions and components.

Several works have addressed the problem of identifying critical components or locations in CTIs, mainly considering their behavior in case of external hazards and threats [29], [30], such as cyber [23] and terroristic attacks [49], [52], [71], and natural disasters [51].

Hausken [29] developed a generic methodology to analytically analyze interdependencies among complex infrastructures, based on defense-attack models and risk analysis. Wu, et al. [71] proposed an attack-strength degradation model, based on network theory, to capture serious physical and geographical interdependencies causing cascading failures among different infrastructures subjected to touristic attacks. Chopra and Khanna [15] developed a framework, based on graph and network theories, for identifying interdependencies among sectors of different infrastructures that amplify the impact of natural disasters on the resilience of the US economy. Genge, et al. [23] suggested a methodology, which relies on sensitivity analysis and the knowledge of the system dynamics, for the identification of those control variables which are critical with respect to cyber-attacks to infrastructures. Patterson and Apostolakis [52] presented a procedure for the identification of critical geographical sectors in dependent infrastructures, based on network theory combined with Monte Carlo simulation to estimate importance measures of the sectors.

A common difficulty of the application of these methods to CTIs is that they require high level of knowledge of the systems, such as the CTI architecture, dynamic model, structure function, which is not easy to retrieve for complex and evolving systems [6], [10], [67], [75], [77] In this context, we propose an approach for critical components identification, based on the analysis of the operational data collected from the CTI monitoring systems. These data are the values of tens of thousands of signals recorded over long periods of time and under different CTI operational setting [8], [74] and, the information on the component criticalities is mined out from these data.

We consider the identification of components critical for the CTI production availability. The objective is to identify the systems bottlenecks to which allocate analysis and consolidation efforts, with ultimate expected benefits in terms of reduction of the probability of abnormal functional conditions, number of CTI shut downs and time of recovery [56], [72]

For this, we develop a novel method based on binary classifiers, which associate monitored signal values to the CTI operating or failed states. The CTI critical components identification problem becomes a feature selection problem for identifying the subsets of measured signals which provide the best classification of the CTI states [9], [17], [27], [45], [46] The components are, then, identified as critical from the set of signals which optimally classifying the CTI states.

Feature selection methods can be typically classified into three categories: filter, wrapper and embedded methods [5], [13], [17], [33], [39]. In filter methods, the feature selector algorithm is independent of the specific learning algorithm used for classification. A numerical evaluation function, computed directly from the data, is used to score the alternative feature subset [17] Wrapper methods search for a subset of features that maximizes the accuracy for the particular algorithm used for the final classification [76] [20], [36]. Embedded methods, such as LASSO [64], follow a scheme for automatically removing features during model training by, for instance, introducing a penalty for the magnitude of the model coefficients. Wrapper methods allow developing more accurate classifiers than filter methods, thanks to their capability of selecting the features most suitable for the specific classification algorithm used [39], [66]. In this work, we develop a wrapper approach for selecting features (signals) that allow best classifying the state of a CTI (functioning or failed) and through which we can identify the most critical components of the CTI. It is based on Binary Differential Evolution (BDE) [47] for the optimization of Support Vector Machine (SVM) CTI state classification. The BDE evolves a population of solutions (feature subsets) which are evaluated with respect to the CTI state classification performance achieved by the SVM classifiers built on the basis of those feature subsets. BDE has been chosen because of its superiority to greedy search strategies, such as forward selection or backward elimination, in the exploration of the search space of feature subsets [27], [31], [41], [44], [58], [78]. SVM classifiers has been chosen as well-performing classification algorithm with affordable computational cost and few hyperparameters to be [42], [46], [53], [69]. As, CTIs are high reliable systems, most instances of the monitored signals correspond to functionality state of operation (majority class events), whereas very few correspond to CTI failure events (minority class events). For such imbalanced datasets, the Cost Sensitive version of the SVM (CS-SVM) has been considered [42], [62].

In the wrapper approach here proposed, the optimal features are chosen based on multi-objective perspectives, and subsets of signals in the obtained Pareto front are evaluated considering their CTI state classification performances. Then, the components which the Pareto front subset signals refer to are identified as critical.

The novelty of the proposed methodology is twofold: 1) the formulation of the problem of identifying critical components of a system (a CTI, in the specific case of interest) as a feature selection problem for system state classification, 2) the tailoring of the traditional feature selection methods to deal with operational data collected from a CTI.

To show the effectiveness of the proposed approach, its performance is compared to that of other feature selection approaches, considering benchmarks characterized by very large number of features and very imbalanced datasets [42]. Then, the approach is applied to a real dataset collected from the CERN Large Hadron Collider (LHC) particle accelerator, which is a large and complex system of systems whose components failures impact directly on the accelerator performance and overall availability for the physics experiments [11], [50]. Since the LHC accelerator downtime associated to equipment faults is currently around 10% of the operation tim [65] the identification of the critical components is of paramount importance to guide the implementation of strategies for improving the CTI infrastructures performance.

The rest of the work is organized as follows. Section 2 formulates the problem of critical components identification in CTIs. Section 3 describes the proposed approach based on wrapper feature selection. Section 4 validates the proposed feature selection approach using different classification benchmarks. Section 5 describes the case study concerning the identification of the critical components of the CERN LHC and shows the obtained results. Finally, Section 6 draws the final remarks.

Section snippets

Problem Statement

We consider a CTI made of a large number, M, of components. We represent the operating or failed state of the CTI at time t using a Boolean variable:xCTI(t)={1iftheCTIisfailed0iftheCTIisoperating

The generic i-th component of the CTI can be in one of a finite set of exhaustive and mutually exclusive states, e.g. operating at full load, operating at half load, on maintenance, faulty, etc. The component state at time t is represented by the discrete variable xi(t), i = 1,…, M, whereas the

Methodology

The problem of identifying the set of critical components c* is connected to that of identifying the set of signalsy*=(y1*,y2*,,yr*,,yr*) with R < N and yr*{y1,y2,,yn,,yN}, which are most informative for defining the CTI state xCTI. We consider a set of signals y1 more relevant than a set of signals y2 if the classification performance of a classifier G*1(y1):xCTI=G*1(y1), built using the historical instances (y1(k), xCTI(k)), k=1,K, is superior to that of a classifier G*2(y2):xCTI=G*2(y2)

Validation of the wrapper feature selection approach

Classification benchmarks characterized by imbalanced datasets have been taken from the KEEL repositor [1] and used for the validation of the CS-SVM classification model and the feature selection approach proposed in this paper. Table 1 reports the characteristics of the considered datasets.

The performances of the CS-SVM classification model are compared to those of other two SVM-based approaches for imbalanced datasets used in [42]: Random Under Sampling (RUS) and Synthetic Minority

Case study: the CERN LHC electrical network

CERN's Technical Infrastructure (TI) is a large and complex system of systems whose objective is to provide the services required for the operation of its particles accelerators and experimental areas. Since abnormal conditions caused by component malfunctioning, failures or external events can impact directly on the accelerators’ performance and overall availability for the physics experiments, CERN's technical infrastructure is considered critical ([56] [12], [24] This case study considers

Conclusions

CTIs are complex systems that evolve in time structurally and functionally. Identifying the critical components of a CTI is fundamental for taking the proper action to guarantee reliability. However, the identification task is very difficult, due to the system complexity.

In this work, we have developed a method for the identification of critical components in a CTI using the operational data collected from monitoring systems. The problem has been originally addressed as a feature selection

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (78)

  • B. Genge et al.

    A system dynamics approach for assessing the impact of cyber attacks on critical infrastructures

    Int J Crit Infrastruct Prot

    (2015)
  • J. Johansson et al.

    Reliability and vulnerability analyses of critical infrastructures: comparing two approaches in the context of power systems

    Reliab Eng Syst Saf

    (2013)
  • G. John et al.

    Irrelevant features and the subset selection problem

  • R. Khushaba et al.

    Feature subset selection using differential evolution and a statistical repair mechanism

    s.l.:Expert Syst Appl

    (2011)
  • K. Kira et al.

    A practical approach to feature selection

  • R. Kohavi et al.

    Wrappers for feature subset selection

  • W. Kröger

    Critical infrastructures at risk: A need for a new conceptual approach and extended analytical tools

    Reliab Eng Syst Saf

    (2008)
  • J. Liu et al.

    A SVM framework for fault detection of the braking system in a high speed train

    Mech Syst Sig Process

    (2017)
  • Y. Li et al.

    Non-dominated sorting binary differential evolution for the multi-objective optimization of cascading failures protection in complex networks

    Reliab Eng Syst Saf

    (2013)
  • S. Maldonado et al.

    A wrapper method for feature selection using Support Vector Machines

    s.l.:Inf Sci

    (2009)
  • S. Maldonado et al.

    Feature selection for high-dimensional class-imbalanced data sets using Support Vector Machines

    s.l.:Inf Sci

    (2014)
  • S. Patterson et al.

    Identification of critical locations across multiple infrastructures for terrorist actions

    Reliab Eng Syst Saf

    (2007)
  • J. Suykens et al.

    Weighted least squares support vector machines: robustness and sparse approximation

    s.l.:Neurocomputing

    (2002)
  • M. Van der Borst et al.

    An overview of PSA Figures of Merit

    s.l.:Reliab Eng Syst Saf

    (2001)
  • B. Wheeler et al.

    High-resolution alignment of action potential waveforms using cubic spline interpolation

    J Biomed Eng

    (1988)
  • B. Wu et al.

    Modeling cascading failures in interdependent infrastructures under terrorist attacks

    Reliab Eng Syst Saf

    (2016)
  • E. Zio et al.

    Feature-based classifier ensembles for diagnosing multiple faults in rotating machinery

    s.l.:Appl Soft Comput

    (2008)
  • E. Zio et al.

    Optimization of the inspection intervals of a safety system in a nuclear power plant by multi-objective differential evolution (MODE)

    Reliab Eng Syst Saf

    (2011)
  • Alcalá-Fdez, J. et al., 2005. KEEL: Knowledge Extraction based on Evolutionary Learning. [Online] Available at:...
  • P. Baraldi et al.

    Hierarchical k-nearest neighbours classification and binary differential evolution for fault diagnostics of automotive bearing operating under variable conditions

    s.l.:Eng Appl Artif Intell

    (2016)
  • J. Bins et al.

    Feature selection from huge feature sets

  • L. Birnbaum

    On the importance of different elements in a multi-element system

    (1969)
  • J. Boardman et al.

    System of systems - the meaning of OF

  • V. Bolon-Canedo et al.

    A review of feature selection methods on synthetic data

    s.l.:Knowl. Inf Syst

    (2013)
  • CERN, 2016. LHC Brochure....
  • S. Chikhi et al.

    Reliefmss: a variation on a feature ranking relief algorithm

    s.l.:Int J Bus Intell Data Min

    (2009)
  • N. Christianini et al.

    An introduction to support vector machines and other kernel-based learning methods. s.l.

    (2000)
  • E.L. Droguett

    Variable selection and uncertainty analysis of scale growth rate under pre-salt oil wells conditions using support vector regression

    Proc Inst Mech Eng, Part O

    (2015)
  • R. Florez-Lopez

    Reviewing relief and its extensions: a new aproach for estimating attributes considering high-correlated features

  • Cited by (2)

    View full text