A novel deep learning method based on attention mechanism for bearing remaining useful life prediction

Highlights

• A novel neural network for bearing remaining useful life prediction is proposed.

• This end-to-end method with attention mechanism required little prior knowledge.

• Useful degradation information can be mined from long historic data.

Abstract

Rolling bearing is a key component in rotation machine, whose remaining useful life (RUL) prediction is an essential issue of constructing condition-based maintenance (CBM) system. However, recent data-driven approaches for bearing RUL prediction still require prior knowledge to extract features, construct health indicate (HI) and set up threshold, which is inefficient in the big data era. In this paper, a pure data-driven method for bearing RUL prediction with little prior knowledge is proposed. This method includes three steps, i.e., features extraction, HI prediction and RUL calculation. In the first step, five band-pass energy values of frequency spectrum are extracted as features. Then, a recurrent neural network based on encoder–decoder framework with attention mechanism is proposed to predict HI values, which are designed closely related with the RUL values in this paper. Finally, the final RUL value can be obtained via linear regression. Experiments carried out on the dataset from PRONOSTIA and comparison with other novel approaches demonstrate that the proposed method achieves a better performance.

Introduction

As one of the most critical components, rolling element bearing is widely used to evaluate the statement of rotating machine. Any unexpected bearing failures would deflect the mechanical system from normal state, such as precision loss, productivity reduction and even the increase of safety risks [1], [2], [3], [4], [5]. To employ condition-based maintenance (CBM) strategy on rotating machine to avoid such disaster, bearing remaining useful life (RUL) estimation is one of the major tasks.

Generally, RUL prediction methods could be grouped into two main categories, i.e. model-based approaches and data-driven approaches [6]. The core of model-based approaches is to build a mathematical model to accurately describe the degradation of machinery. However, the constructing process requires not only the parameters of the real engineering system after a series of measurements, but also extensive prior knowledge about the systems. Actually, few machineries can be accurately simulated by simple mathematical models for the complexity of real world. Nowadays, model-based approaches for predicting a general trend of the degradation of machinery are still helpful, including particle filter [7] , Eyring model [8] , Weibull distribution [9], etc. On the other hand, data-driven methods attempt to learn the machinery degradation patterns based on historical collected data. Thanks to the strong learning ability, these methods are capable of revealing the underlying correlations and causalities between the phenomenon (the collected data) and the reason (the statement of the corresponding system), especially the complex ones that hardly can be described by manual mathematical model. Because of this characteristic, more and more data-driven approaches in the field of machinery prognostics are proposed and achieved better prognostic results. A comprehensive review of statistical data-driven approaches is presented by Si et al. [10]. Lei et al. [11] built a data-driven model based on artificial neural network to predict bearing RUL. In [12], Huang et al. suggested that a novel health indicator (HI) called minimum quantization error obtained by a self-organizing map (SOM), is then helpful to train back propagation neural networks for degradation prediction model. Chen et al. [13] developed a prognostic method using adaptive neuro-fuzzy inference systems and high order particle filtering. Loutas et al. [14] proposed a data-driven approach for bearing RUL estimation based on support vector regression (SVR), which utilizes multiple statistical features from time-domain, frequency domain and time-scale domain. And more data-driven approaches are introduced briefly in a recent review paper [15].

There is no doubt that, when faced with enormous amount of collected data, data-driven approaches based on deep learning (DL) show more effective processing capacity and achieve more excellent performance, especially in the field of computer vision, natural language processing, speech processing, etc. [16], [17], [18]. And some deep learning techniques have already found their way into machine health monitoring systems. Zhu et al. proposed a stacked autoencoder (SAE) based DNN (Deep Neural Network) for hydraulic pump fault diagnosis that uses frequency features generated by Fourier transform [19]. Liu et al. uses normalized spectrum generated by short-time Fourier transform (STFT) of sound signal as inputs of a 2-layer SAE based DNN. Some researchers [20], [21] feed multi-domain statistical features including time domain features, frequency domain features and time-frequency domain features into SAE as a way of feature fusion. Zhang et al. [22], [23] proposed a novel convolution neural network (CNN) to make a successful bearing fault diagnosis directly on vibration signals. Furthermore, other difficult bearing fault diagnosis problems can be solved by more complex CNNs [24], [25], [26]. Although more and more DL-based approaches are employed to deal with fault diagnose problems in mechanical system, few successful cases can be found in addressing prognosis problem. Recurrent neural network (RNN), instead of CNN, is a smart choice for seeking underlying knowledge from historical data. Malhi et al. [27] proposed a competitive learning-based approach based on RNN for long-term prognostics of machine health statement, where vibration signals collected from a defected rolling bearing are pre-processed with the continuous wavelet transform and used as the model input. Besides, a long short-term memory (LSTM) based neural network scheme was proposed by Yuan et al. [28] for RUL estimation of aero-engines, in the cases of complicated operations, hybrid faults and strong noises. LSTM was also utilized by Zhao et al. [29] for a tool wear health monitoring task.

Though many works mentioned above have achieved good results, few of them are purely data-driven. In order to make an accurate prognosis, two main steps are still essential: health indicator calculation and bearing RUL prediction. For now, manual methods for calculating health indicator are still the most popular and common ones, because such selected indicator has an obvious trend to make it easier to predict the RUL of bearing. But this still requires much expert knowledge on mechanical system, degradation theory and statistics. Besides, data-driven methods have always been employed in the second step to regress the degradation curve of health indicator, but the failure point is still calculated according to an empirical special threshold in general.

Furthermore, accurate and effective prognosis in mechanic system is still hampered by a vital drawback of RNN. The collected data responding to the whole life of the machine is so long that RNN failed to process it, due to the so-called vanishing and exploding gradient problems [30]. To the authors’ best knowledge, most of proposed RNN-based models for prognosis make do with sliced data instead of the whole long data [31], which may cause some problems such as overfitting and decrease the prognosis accuracy.

In order to solve the aforementioned shortcoming, this paper proposes an RNN model based on encoder–decoder structure with attention mechanism. First, five band-pass energy values of frequency spectrum of vertical and horizontal vibration signals are used as input to train and test the proposed RNN model. Different from traditional RNN structure, attention machine is added to decide the attention distribution according to the first look by encoder, which help decoder make a better prognosis and overcome the vanishing and exploding gradient problems. As a result, a sequence of HI values ranges from 0 to 1 is supposed to be obtained. Finally, by linear regression with least square method, the accurate prediction of bearing RUL can be calculated. The main contributions of this paper are summarized as follows.

• A novel method with little prior knowledge for rolling bearing RUL prediction is proposed, and achieved high prognosis accuracy.

• An RNN model based on encoder–decoder structure with attention mechanism is proposed to mine useful degradation information from long historic data, which is validated by visualizing the attention distribution.

The rest of this paper is arranged as follows. Section 2 introduces the basic theories of RNN, LSTM, GRU (Gated Recurrent Unit), encoder–decoder structure and attention mechanism. The detailed process of proposed approach is then described in Section 3. In Section 4, our method is validated using the dataset from the accelerated degradation testing on rolling element bearings. Furthermore, the analysis of the proposed neural network is also presented to explain its effectiveness. Finally, conclusions are drawn in Section 5.