VisPro: a prognostic SqueezeNet and non-stationary Gaussian process approach for remaining useful life prediction with uncertainty quantification

Abstract

Rotating machinery is essential to modern life, from power generation to transportation and a host of other industrial applications. Since such equipment generally operates under challenging working conditions which can lead to untimely failures, accurate remaining useful life (RUL) prediction is essential for maintenance planning and prevention of catastrophic failures. In this work, we address current challenges in data-driven RUL prediction for rotating machinery. The challenges revolve around the accuracy and uncertainty quantification of the prediction, and the non-stationarity of the system degradation and RUL estimation given sensor data. We devise a novel computational architecture and RUL prediction model with uncertainty quantification, termed VisPro, which integrates time–frequency analysis, deep learning image recognition, and nonstationary Gaussian process regression. We analyze and benchmark the results obtained with our model against those of other advanced data-driven RUL prediction models using the PHM12 bearing vibration dataset. The computational experiments show that (1) the VisPro predictions are highly accurate and provide significant improvements over existing prediction models (three times more accurate than the second-best model), and (2) the RUL uncertainty bounds are valid and informative. We identify and discuss the architectural and modeling choices made that explain this predictive performance of VisPro.

Appendices

Appendix A: Examining the effectiveness of NSGPR

In this appendix, we examine the effectiveness of the NSGPR in our RUL prediction model by comparing the RUL prediction with and without NSGPR. The RUL predictions without NSGPR are taken from the last step prediction of the Pro-SQM network in the testing dataset. The RUL prediction without NSGPR the testing Bearing 1_2 is shown in Fig. 

Fig. 11

The RUL estimation of the DL network of the testing Bearing 1_2

11.

First, we note that the RUL estimation curve is oscillating more significantly compared with the RUL prediction with NSGPR as shown in Fig. 9. Second, the RUL prediction is 1360 s, which has a larger error than that of the prediction with NSGPR. The RUL predictions without NSGPR of overall testing Bearings are shown in Fig. 

Fig. 12

The RUL estimation of all Bearings in the testing set of DL network

12.

First, the RUL prediction in Fig. 12 does not provide the lower and upper bounds, and the use of NSGPR supports our model with uncertainty quantification. Second, in this work, we use the scoring function as calculated in Eq. 9 to measure the accuracy of the model prediction. The scoring function for RUL predictions without NSGPR is 0.59. The scoring function of VisPro predictions is 0.84, which is 42% higher than that of the predictions without NSGPR.

Appendix B: Structure and weight parameters of the Pro-SQN

Here, we introduce the details of the output size, memory requirement per image, and the number of the weight of Pro-SQN as shown in Table

Table 5 The output size, hardware memory, and weight size of the Prognostic-SqueezeNet discriminator

5. We use 32-bit floating numbers for variables in the data processing that one variable takes $\frac{32}{8}=4$ bytes. According to Table 5, the number of weights is 1,187 M and it possesses 0.594 Mb on memory for the model itself. The memory usage is the hard requirement of the hardware and weight size indicates the training requirement of the model.

Then, we introduced the fire model as shown in Fig. 

Fig. 13

Organization of convolution filters in the fire model. In this example, ${{\varvec{s}}}_{1{\varvec{x}}1}=3$, ${{\varvec{e}}}_{1{\varvec{x}}1}=4$, and ${{\varvec{e}}}_{3{\varvec{x}}3}=4$. We illustrate the convolution filters but not the activations [33]

13. 1 since it is extensively used in the SqueezeNet, where ${s}_{1x1}$, ${e}_{1x1}$, and ${e}_{3x3}$ stand for the number of squeeze layers, the number of $1\times 1$ expand layer, and the number of $3\times 3$ expand layers, respectively.

In our fire detection SqueezeNet, we set ${s}_{1x1}$, ${e}_{1x1}$, and ${e}_{3x3}$ as 1. We switch the activation function of the fire model from ReLU in the original model to LeakyReLU for more nonlinearity and preventing vanishing gradient problem for negative input.

Appendix C: Examining the effectiveness of the local length scale kernel

In this appendix, we examine the effectiveness of the local length scale kernel in our RUL prediction model. In order to demonstrate the effectiveness of the local length scale, we compare our results with a prediction with dot product and squared exponential kernel, which has a universal length scale. The RUL predictions without local length scale kernel of overall testing Bearings are shown in Fig. 

Fig. 14

The RUL estimation of the entire Bearings in the testing set without local length scale kernel

14 and Table

Table 6 Comparison of RUL prediction of VisPro with and without local length scale kernel

6.

First, in Table 6, the RUL prediction with local length scale kernel has a significant advantage over the prediction with squared exponential kernel. The mean of the prediction error, STD of the error, and score function are improved by 66%, 55%, and 14%, respectively. Second, in Table 6, the RUL prediction error of Bearing 2_4 is significantly improved from 27.34 to 7.91 by using local length scale kernel. In order to investigate the details of this improvement, the RUL predictions for the testing Bearing 2_4 with and without local length scale kernel are compared in Fig. 

Fig. 15

The RUL estimation of Bearing 2_4 a with local length scale kernel; b without local length scale kernel

15.

Comparing the results of RUL prediction of Bearing 2_4 with local length scale kernel and without local length scale kernel (with squared exponential kernel), first, the prediction of without local length scale kernel is oscillating. Since the local length scale kernel considers the local smoothness, its prediction is less oscillating and more robust. This consequently improves the prediction accuracy of the B 2_4 in the testing dataset. Second, in Fig. 15, the uncertainty bound is larger after truncation time (6110 s) for the prediction without local length scale kernel compared with that of the prediction with local length scale kernel. Consequently, the use of local length scale kernel in the NSGPR step provides a more precise RUL prediction with a tighter uncertainty bound.

Appendix D: Uncertainty quantification of 80%, 90%, and 95% confidence interval

In this appendix, we examine the uncertainty quantification results with 80%, 90%, and 95% confidence intervals. The prediction and uncertainty quantification results are shown in Fig. 

Fig. 16

The RUL estimation of the entire Bearings in the testing set with 80%, 90%, and 95% confidence interval

16.

Figure 16 shows that first the mean estimation of RUL is identical and does not vary with different confidence intervals. Second, virtually, the size of uncertainty shrinks with the decrease of its percentage. For example, 80% confidence interval is tighter compared with that of 90% and 95%. However, the uncertainty quantification has potential invalid cases that ground truths are outside the uncertainty bound for the 80% confidence interval. We calculate and summarize the average confidence interval size and invalid case number among all testing bearings for 80%, 90%, and 95% confidence intervals in Table

Table 7 The uncertainty quantification performance with 80%, 90%, and 95% confidence interval

7.

Table 7 shows 80% confidence interval has the tightest uncertainty bound with the average size of 322 s. However, it has 3 invalid cases that the real RULs are out of the estimated uncertainty bounds, and their uncertainty quantifications are not informative. Second, 90% has a tighter uncertainty bound compared with that of 95% without invalid case of uncertainty quantification. In this way, with the considerations of both average size and invalid case of the uncertainty quantification, 90% confidence interval is the optimal selection for VisPro. We use 90% confidence interval in Sect. 4 to discuss the RUL prediction and uncertainty quantification results.