Monitoring techniques for an online neuro-adaptive controller

Abstract

The appeal of biologically inspired soft computing systems such as neural networks in complex systems comes from their ability to cope with a changing environment. Unfortunately, adaptability induces uncertainty that limits the applicability of static analysis to such systems. This is particularly true for systems with multiple adaptive components or systems with multiple types of learning operation.

This work builds a paradigm of dynamic analysis for a neuro-adaptive controller where different types of learning are to be employed for its online neural networks. We use support vector data description as the novelty detector to detect unforeseen patterns that may cause abrupt system functionality changes. It differentiates transients from failures based on the duration and degree of novelties. Further, for incremental learning, we utilize Lyapunov functions to assess real-time performance of the online neural networks. For quasi-online learning, we define a confidence measure, the validity index, to be associated with each network output. Our study on the NASA F-15 Intelligent Flight Control System demonstrates that our novelty detection tool effectively filters out transients and detects failures; and our light-weight monitoring techniques supply sufficient evidence for an insightful validation.

Introduction

Adaptive systems are those systems whose functionality evolves over time in response to changes in the environment in which it operates. If learning and adaptation are allowed to occur after the system is deployed, the system is called online adaptive system Mili et al. (2003). The use of biologically inspired soft computing systems for online adaptation to counter the changes in system environments has revolutionized the operation of real-time automation and control applications. Neural networks are one of the most popular learning paradigms employed in online adaptive systems. Because the learning algorithms behind these computational architectures are usually derived from error/risk minimization theories, the computations are complex and the learning process is non-linear.

Applications of adaptive computing in safety critical systems are on the rise. These applications provide fault-tolerant control capabilities, automated maintenance of distributed networks, or enhance implementations of high security devices. Different research communities use different terms to describe online adaptive computing. For example, in computer networks automated modification of internal variables in traffic routing, possibly through judicious application of machine learning algorithms, is called the parameter adaptation paradigm McKinley et al. (2004). The Flight Control has become one of the most promising emerging applications of real-time adaptive control. The mechanisms and algorithms for achieving fault tolerant system features through adaptivity are termed software-enabled control, adaptive control augmentation or intelligent flight control.

When learning software is used in online adaptation, its behavior has direct consequence on the performance of systems into which it is embedded. Therefore, it is necessary to understand, predict and assure the behavior of the adaptive learner before its actual deployment. And while many experiments, especially in the aerospace domain, have demonstrated the potential of online adaptive computing, the lack of verification and validation procedures represents a serious problem barring their widespread use.

Our previous research using formal methods on certain families of neural networks suggests that environmental changes (learning data) have a significant impact on system behavior Mili et al. (2003). Abrupt and abnormal environmental changes are likely to generate previously unseen learning behaviors, because judicious enumeration of all environmental conditions is impossible. Therefore, we believe that the most promising paradigm for validating the performance of adaptive systems is their continual monitoring. The monitoring data collected in real time is then used to analyze system properties of interest, most typically related to convergence and stability of the learning algorithm Yerramalla et al. (2003).

Therefore, the most meaningful questions related to the monitoring approaches to validation of online adaptive systems are the following two:

• What to monitor?

• How to analyze monitored data?

This paper provides our answers to these questions. The data stream entering the adaptive computational element, such as the neural network, is monitored to detect anomalous data patterns which indicate significant environmental changes. The internal structure of the neural network is monitored and analyzed with respect to the convergent behavior that usually indicates successful learning. Furthermore, the network predictions, i.e., its output data stream, is monitored and statistically analyzed for variations as an indication of their volatility over time.

Our research further indicates that the choice of the methods for system validation is phenomenological. In other words, we encountered systems where, depending on the design choices, not all the above mentioned techniques are feasible. Therefore, while these techniques are complementary, the feasibility of their application in the analysis of adaptive learning varies. This paper presents three monitoring and analysis techniques, demonstrates their partial applicability to two types of learning modes, both investigated in the context of an experimental adaptive flight control scheme.

The Intelligent Flight Control System (IFCS) was developed by NASA with the primary goal to “flight evaluate control concepts that incorporate emerging soft computing algorithms to provide an extremely robust aircraft capable of handling multiple accident and/or an off-nominal flight scenario” Boyd et al., 2001, Jorgensen, 1991. The diagram in Fig. 1 shows the architectural overview of NASA’s IFCS implementation using Online Learning Neural Network (OLNN). The control concept can be briefly described as follows. Notable discrepancies from the outputs of the Baseline Neural Network and the Real-time Parameter Identification (PID), either due to a change in the aircraft dynamics (loss of control surface, aileron, stabilator) or due to sensor noise/failure, are accounted by the OLNN.

The primary goal of OLNN is to accomplish in-flight accommodation of discrepancies, commonly known as Stability and Control Derivative errors. Derivative errors indicate conditions that fall outside the scope of traditional (linearized) control gain look-up tables. When OLNN performs adaptation, its behavior has a direct consequence on the performance of the flight control system. In such a safety-critical application, it is necessary to understand and predict the behavior of the NN. The goal of validating NN-based online component is to provide a means to detect novel (abnormal) conditions entering the OLNN, to investigate the NN’s stability behavior during adaptation, and to interpret the results of the analysis so that it ensures safe operation.

One concern in this process is the presence of noise in sensor data as well as temporary variations in environmental data which we may wish the neural network to tolerate without disruption of the training process. In any validation method, it is necessary to distinguish between such transient disturbances which do not persist or may not impact performance and true failures which do persist or have a major impact on the system. The transient nature of a disturbance may not become apparent except over time and the true impact may only emerge after an adaptive learner has been trained, but it is still necessary to respond to extreme variability as quickly as possible. The primary contribution of the current work is to observe that an integrated approach to validation is needed in which monitoring occurs at multiple time scales so that both immediate analysis of real-time learning and buffered long-term failure indicators are used.

The rest of the paper is organized as follows. In Section 2 we give a summary of the adaptive component of the IFCS. This section also reviews of the underlying structure of the neural network, describes the flight simulator used to generate test data and lists the failure modes used in the ensuing experiments. Section 3 introduces novelty detection techniques which discriminate between permanent failure events and transient failure events before their data representations reach the input layer of the neural network. The same section discusses the accuracy of this method. Sections 4 Validating Type I online learning, 5 Validating Type II online learning outline two techniques for analyzing and validating the real-time behavior of the neural network used by the IFCS. Each technique addresses a different type of the learning algorithm. We also describe experimental results by applying the techniques to two simulated failure modes. Finally, we conclude with a summary and list our future research directions in Section 6.