Gated recurrent unit based recurrent neural network for remaining useful life prediction of nonlinear deterioration process

Highlights

• A general solution is presented for RUL prediction of nonlinear deterioration process.

• KPCA is selected for dimensionality reduction and nonlinear feature extraction.

• GRU is presented to replace LSTM, which behaves better both in prediction accuracy and training time.

Abstract

Remaining useful life (RUL) prediction is a key process for prognostics and health management (PHM). However, conventional model-based methods and data-driven methods for RUL prediction are bad at a very complex system with multiple components, multiple states and therefore extremely large amount of parameters. In order to solve the problem, a general two-step solution is proposed in this paper. In the first step, kernel principle component analysis (KPCA) is applied for nonlinear feature extraction. Then, a novel recurrent neural network called gated recurrent unit (GRU) is presented as the second step to predict RUL. GRU network is capable of describing a very complex system because of its specially designed structure. The effectiveness of the proposed solution for RUL prediction of a nonlinear degradation process is proved by a case study of commercial modular aero-propulsion system simulation data (C-MAPSS-Data) from NASA. Results also show that the proposed method requires less training time and has better prediction accuracy than other data-driven methods.

Introduction

PHM is a basic requirement for condition-based maintenance in many application domains where safety, reliability, and availability of the systems are considered mission critical [1]. Particularly, RUL prediction is one of the main tasks in PHM. Improving the accuracy of the RUL prediction can not only enhance the safety and reliability, but also prolong service time which decreases the average cost in turn. Therefore, many researchers have studied RUL prediction methods in recent years.

Generally, there are two approaches for RUL prediction: model-based methods and data-driven methods. Model-based methods can be used for a component or a simple system to deduct a more accurate RUL by building a physical failure model while data-driven methods can estimate RUL for a complex system by constructing a simpler data-based model. In order to predict RUL of complex systems, data-driven methods therefore has got more attention recently [2]. Ahmad et al. [3] predicted the RUL of the rolling element bearings using dynamic regression models. Hu et al. [4] proposed a prediction method for the RUL of wind turbine bearings based on the Wiener process. Huang et al. [5] presented an adaptive skew-Wiener process model for RUL prediction. Zhang et al. [6] presented a review on Wiener-process-based methods for RUL prediction and degradation data analysis. Le et al. [7] estimated the RUL with noisy gamma deterioration process. Ling et al. [8] proposed Bayesian and likelihood inferences on remaining useful life in two-phase degradation models under gamma process. Baptista et al. [9] proposed a method for RUL prediction combining data-driven and Kalman filter. Son et al. [10] predicted the RUL based on noisy condition monitoring signals using constrained Kalman filter. Duong et al. [11] presented a method with heuristic Kalman optimized particle filter for RUL prediction. Liu et al. [12] proposed a novel method using adaptive hidden semi-Markov model (HSMM) for multi-sensor monitoring equipment health prognosis. Chen et al. [13] presented a hidden Markov model (HMM) with auto-correlated observations for RUL prediction and optimal maintenance policy. Li et al. [14] proposed an optimal Bayesian control policy for gear shaft fault detection using HSMM. Chen et al. [15] proposed a general solution to nonlinear multistate deterioration modeling with non-homogeneous hidden semi-Markov model (NHSMM) for deterioration level assessment and RUL prediction. Moghaddass et al. [16] presented an integrated framework for online diagnostic and prognostic health monitoring using a multistate deterioration process. Although these methods are widely used, they have their own limitations. The deterioration process of the equipment is usually nonlinear and multiple, because of the complicated structure and variable work status. The deterioration curve may not follow a typical shape such as exponential or linear function. It is an important challenge of RUL prediction that finding out the rule of the nonlinear deterioration. Wiener process, Gamma process and Kalman filter perform not very well when the deterioration process is nonlinear. HMM model performs well on nonlinear deterioration process but the training time increases dramatically when multiple system states are concerned.

Another important branch of data-driven methods is artificial intelligence (AI). In recent years, AI, particularly deep learning methods, has achieved outstanding performance in image processing, natural language processing (NLP) and so on. Researchers have also exploited applications of AI methods for RUL prediction. Among deep learning methods, recurrent neural network (RNN) has attracted special attention because its network structure contains recurrent hidden layer, which is very suitable for time series processing and consequently RUL prediction. Guo et al. [17] proposed a recurrent neural network based health indicator for RUL prediction of bearings. Liu et al. [18] proposed a method for fault diagnosis of rolling bearings with recurrent neural network-based auto-encoders. However, RNN cannot link two similar data if they are separated too far away.

In order to overcome the weakness of RNN, long short term memory (LSTM) is proposed, which introduces input gate, output gate and cell state into RNN [19]. LSTM could save long-time memory into cell state and it has been verified as a most mature and efficient method on many tasks. Hinchi and Tkiouat [20] proposed a method based on LSTM for RUL prediction of rolling bearing. Yuan et al. [21] proposed a method for RUL prediction of aero engine using LSTM neural network. Malhotra et al. [22] proposed a method for multi-sensor prognostics by using an unsupervised health index based on LSTM encoder-decoder. However, each memory blocks in LSTM needs an input gate and an output gate. These gates make the training more difficult and increase the training time of the network.

To reduce training time and improve network performance, a simplified but improved LSTM-architecture network, GRU, is proposed [23]. The GRU chooses a new type of hidden unit that merges the forget gate and the input gate into a single update gate and mixes cellular state and hidden state into one state as well. In brief, the number of gates is decreased from 4 in LSTM to 2 in GRU, named update gate and reset gates.

A general two-step solution for RUL prediction of nonlinear deterioration process is proposed to deal with the nonlinearity in deterioration modeling. In the solution, (1) KPCA is applied as the first step for nonlinear feature extraction. By reducing the dimension, over-fitting caused by too many model parameters can be effectively avoided. (2) GRU, a simplified network of LSTM with fewer parameters, is presented to predict RUL. In practice, (3) Sequence-to-one method is applied, increasing the number of samples while avoiding the trouble of variable length sequence input. 4) Sliding average method is applied to smooth the results, increasing the prediction accuracy effectively.

The rest of the paper is organized as follows. Section II describes different RNN structures. In section III, a general two-step solution for RUL prediction is proposed. In Section IV, the C-MAPSS-Data is used to verify the efficiency and accuracy of the proposed method. Finally, conclusion is drawn in Section V.