Adaptive self-attention LSTM for RUL prediction of lithium-ion batteries

Abstract

To achieve an accurate remaining useful life (RUL) prediction for lithium-ion batteries (LIBs), this study proposes an adaptive self-attention long short-term memory (SA-LSTM) prediction model. The innovations of the designed prediction model include the following. (1) It features an optimized local tangent space alignment algorithm, which allows the extraction of an indirect health indicator (HI) that can precisely describe battery degeneration from charge data. The extracted HI exhibits a high correlation with the standard capacity, thus facilitating RUL estimation. (2) By introducing a masked multi-head self-attention module into the time-series prediction model based on LSTM, critical information in the sequences is captured and the prediction performance is improved. (3) An online self-tuning mechanism for the weights and biases of neural networks is designed to correct cumulative estimation errors in long-term predictions and reduce the effects of local fluctuations and regeneration. The proposed prediction model enables the HI values in future cycles to be iteratively estimated using the one-step-ahead method, and the RUL can be forecast once the predicted signal falls. Experimental results indicate the effectiveness and superiority of the proposed prediction method.

Introduction

Lithium-ion batteries (LIBs), as an alternative source of fossil fuel, demonstrate high potential for addressing energy depletion and environmental crises. Therefore, they have been used extensively in electric vehicles, aerospace, consumer electronics, and other fields [1]. However, the performance of LIBs degrades over time; thus, it can result in battery failure or severe accidents. In this regard, an accurate estimation of the remaining useful life (RUL) can significantly facilitate battery performance monitoring, failure warning, and battery replacement, thus helping in avoiding unexpected breakdowns and safety incidents.

However, RUL prognosis remains challenging in practical scenarios. Battery degradation is an unknown and comprehensive nonlinear dynamic process affected by the complex interplay between internal electrochemical reactions and external operating conditions [2], [3]. The nonlinear dynamic process results in intricate degradation phenomena during battery operation, such as accelerated degradation (AD) from the middle to the end of battery life [4] and the local regeneration phenomenon (CRP) [5], which occurs during the resting phase of the battery. Consequently, an appropriate model must be established to accurately describe the degradation patterns and dynamics of LIBs, which is challenging. Additionally, the battery typically has a lifespan of hundreds or even thousands of cycles, which satisfies the practical requirements of long lifetime and high storage [6]. The long lifespan poses a challenge for modeling the long-term dependencies of the battery degradation process. Furthermore, battery capacity and resistance, which are the most typically used direct indicators for RUL prediction, cannot be measured online but can only be measured in a laboratory using specific measuring equipment or operating conditions [7], [8].

To overcome these challenges, this study developed a data-driven RUL prognosis algorithm for LIBs by applying adaptive deep learning. This novel prediction method considers the characterization of LIB degradation and the construction of a long-term prediction model such that an accurate RUL prediction for LIBs can be achieved.

RUL prediction methods can be classified into two primary categories [1]: model-based and data-driven methods.

1) Model-based method.

For the model-based method, a degradation model of a battery must be established based on prior knowledge or physical laws; these models can be further classified into the failure mechanism model (FMM) [9], equivalent circuit model (ECM) [10], filtering model (FM) [11], and stochastic process model (SPM) [12]. To investigate the chemical or physical properties and operating principles of both the FMM and ECM, battery aging analysis must be performed. However, these approaches require complicated modeling processes, high accuracy, and generalization. The aim of the FM and SPM is to mine the recurrence relations of the internal battery status or the change regulations of the battery monitoring data [13]. However, these methods exhibit the disadvantages of low dynamic accuracy, limited adaptability, and insufficient prior knowledge owing to the interaction between internal mechanisms and external operations.

2) Data-driven method.

The data-driven method, also known as the model-free method, directly infers the degradation state and predicts the RUL from battery monitoring data. Thus, this method does not require electrochemical analysis or prior knowledge, which distinguishes it from model-based methods [14]. Recently, various intelligent algorithms, such as vector machines [15], autoregressive modeling [16], and neural networks (NN) [17], have been widely applied for RUL prediction because of their high flexibility and adaptability in approximating nonlinear systems. In particular, advanced deep learning NNs [18], [19] perform remarkably well in nonlinear and high-dimensional data modeling; this demonstrates their potential in data-driven RUL prediction. As an effective method for managing time series, a long short-term memory (LSTM) NN is explicitly designed to control the flow of information for long-term dependencies and eliminate vanishing or exploding gradients [20], [21]. Furthermore, the elaborate gate mechanisms and loop cell states in an LSTM NN guarantee that the predicted state is affected less by backpropagation. This allows critical information to be partially retained longer compared with using the typical recurrent NN (RNN)-based methods. Researchers highly recommend using LSTM and its variants for estimating both the long-term degradation state and the RUL of LIBs [22], [23], [24]. Nevertheless, several limitations exist in the prediction performance of vanilla LSTM-based methods, which are presented as follows.

First, LSTM fails to capture the dynamic degradation properties (i.e., the AD and CRP) in battery time series [25]. Generally, before the LSTM NN performs a prediction, a learning process is required to determine the network parameters using the historical data of LIBs. Because the degradation dynamics of LIBs are unknown and uncertain, parameter-fixed LSTM trained only with historical data demonstrates unsatisfactory adaptability and generalization in the prediction stage [26]. Furthermore, the cumulative errors increase with the number of prediction cycles under the one-step or multistep forward iterative mode of LSTM-based sequence prediction [27]. In particular, when encountering accelerated degradation or local regeneration, the predicted value may deviate promptly from its actual value, thus resulting in a rapid augmentation of cumulative errors and the possible failure of iterative prediction. To avoid these issues, researchers [5], [22], [28], [29] have attempted to separate the local regeneration of time series from a global trend by employing signal decomposition methods. Nevertheless, these signal-separated prediction methods require the construction of the corresponding submodels for all decomposed subsignals, which significantly increases the computational cost and uncertainty of RUL prediction.

Second, although LSTM can alleviate the effect of long-term dependencies by emphasizing the dependencies of a sequence on its proximity in a timely manner, relatively more effort is required for LIBs. This is because the contributing features of battery degradation are not in chronological order, and some arbitrary early steps may contribute to the final RUL prediction [30]. An efficient operation requires the prioritization of more important features or time steps by assigning larger weights [31]. The attention mechanism was explicitly proposed for assigning weights to prompt a model to focus on more important features, regardless of their distance in the sequence [32]. In this regard, the self-attention (SA) module, which is the key module of the transformer architecture, has been demonstrated to be a state-of-the-art method [33], [34]. It describes the global dependencies between inputs and outputs with high parallelization and computational efficiency. The effectiveness of SA in improving prediction has been proven in the RUL prediction of mechanical components when it was combined with an RNN [3], [35]. Nevertheless, the prediction performance requires further improvement owing to the widespread long-term dependencies throughout the life of LIBs.

Third, health indicator (HI) extraction is another critical element of data-driven RUL prediction methods. The essence of the data-driven method is the regression of relevant performance indicators. Thus, an HI that can accurately characterize battery degradation is necessary. Recently, researchers have focused on indirect HIs, which can be easily extracted from measurable battery data, such as incremental capacity analysis curves [14], [36] and discharge voltage difference [37]. However, the extraction of these HIs is performed manually, is time consuming, and requires extensive domain knowledge. By contrast, several intelligent algorithms, such as NNs, evolutionary algorithms (EAs), and manifold learning (ML) methods, can extract HIs automatically and efficiently [38]. However, both the NNs (such as the autoencoder (AE) and its extensions) [39] and EAs lack feasible guidelines for parameter tuning and exhibit additional computational complexity. By contrast, ML methods, which offer the advantages of low computational burden and high execution efficiency, are regarded as effective approaches for nonlinear dimension reduction. The main purpose of ML is to enable the construction of nonlinear low-dimensional manifolds using sampled data points in high-dimensional spaces. Typical ML methods include multidimensional scaling (MDS) [40], local linear embedding (LLE) [41], and local tangent space alignment (LTSA) [42]. Notably, LTSA, which is an improved version of LLE for solving the singularity of weight coefficients, offers a higher embedding accuracy, stronger noise resistance, and less overhead compared with other ML methods. However, it requires manual intervention when selecting the local neighborhood size, which is impractical in real-time applications [43].

In summary, constructing a dependable network to estimate the HI and RUL of LIBs in the presence of strong long-term dependencies and dynamic degradation properties (including the AD and CRP) as well as constructing highly reliable indirect HIs remain challenging.