Theoretical study of GDM-SA-SVR algorithm on RAFM steel

Abstract

With the development of society and the exhaustion of fossil energy, we need to identify new alternative energy sources. Nuclear energy is an ideal choice, but the key to the successful application of nuclear technology is determined primarily by the behavior of nuclear materials in reactors. Therefore, we studied the radiation performance of the fusion material RAFM steel. We used the GDM algorithm to upgrade the annealing stabilization process of simulated annealing algorithm. The yield stress performance of RAFM steel was successfully predicted by the hybrid model, which combined simulated annealing with the support vector machine for the first time. The prediction process was as follows: first, we used the improved annealing algorithm to optimize the SVR model after training on a training dataset. Next, we established the yield stress prediction model of RAFM steel. By testing the model and conducting sensitivity analysis on the model, we can conclude that, compared with other similar models such as the ANN, linear regression, generalized regression neural network, and random forest, the predictive attribute variables cover all of the variables in the training set. Moreover, the generalization performance of the model on the test set is superior to that of other similar prediction models. Thus, this paper introduces a new method for the study of RAFM steel.

Appendices

Appendix A: Hessian matrix

The second-order partial derivative of the real value function on the real vector is called the Hessian matrix, which is expressed as H[f(x)]and is defined in the following form:

$$\begin{aligned}{}[Hf(x)]=\frac{\partial f(x)}{\partial x{\partial x^{\top }}}=\frac{\partial \frac{\partial f(x)}{\partial x^{\top }}}{\partial x}\in {\mathbb {R}}^{n\times n}. \end{aligned}$$

(29)

Therefore, the (s, t) element is defined as

$$\begin{aligned} H[f(x)]=\frac{\partial ^2 f(x)}{\partial x \partial x^{\top }}= \begin{bmatrix} \frac{\partial ^2 f}{\partial x \partial x^{\top }}&\cdots&\frac{\partial ^2 f}{\partial x \partial x^{\top }}\\ \vdots&\ddots&\vdots \\ \frac{\partial ^2 f}{\partial x \partial x^{\top }}&\cdots&\frac{\partial ^2 f}{\partial x \partial x^{\top }}\\ \end{bmatrix}\in {\mathbb {R}}^{n\times n}. \end{aligned}$$

(30)

The Hessian matrix is a symmetrical matrix because the second-order partial derivative and the mixed partial derivative have nothing to do with the order of the derivation.

Appendix B: Statistic information of sample data

Refer to Table 4.

Table 4 Basic information of the input parameters

Appendix C: Extreme point discrimination of real variable functions

Theorem 1

IfUis a local extreme point ofF(x), andf(x) is continuously differentiable inP(U, R), the neighborhood of pointU. Then, we have

$$\begin{aligned} \nabla _{u} \,f(u)=\frac{\partial f(x)}{\partial x}|_{x=u}=0. \end{aligned}$$

(31)

This is only the necessary condition for the first-order stationary point, and it is not a sufficient and necessary condition. For example, the saddle point also satisfies the above constraints.

Theorem 2

If the second-order partial derivative is continuous in the open neighborhood ofUand satisfies

$$\begin{aligned} \nabla _u\, f(u)=\frac{\partial f(x)}{\partial x}|_{x=u}=0 \end{aligned}$$

(32)

$$\begin{aligned} \nabla _{u}^{2}\,f(u)=\frac{\partial ^2 f(x)}{\partial x \partial x^{\top }}|_{x=u}\succ 0, \end{aligned}$$

(33)

thenUis a strict local minimum point of a functionf(x). The Hessian matrix of pointUis a positive definite matrix.

Appendix D: Support vector regression model

The main principle of the SVR model is to map data characteristics to high-dimensional space through the kernel function, making many non-separable low dimensional data become linearly separable in high-dimensional Hilbert space (characteristic space). With the support of strong mathematical theory (based on the VC dimensional generalization theory framework and the principle of structural risk minimization) and its superior performance, SVR quickly became mainstream in machine learning. Before deep learning, it was always one of the preferred models in the field of application because of its superior performance.

For the regression problem, a given sample is $\hbox {D}=\{$(x1, y1), (x2, y2), … $\}$, and a model is expected to be learned as follows:

$$\begin{aligned} \underset{\alpha }{max}J(\alpha )=\frac{\alpha ^{\top }M \alpha }{\alpha ^{\top }N \alpha }. \end{aligned}$$

(34)

Other regression models often use the difference between the output f(x) and the real value to calculate the loss (for example, mean square error); however, SVR has an $\epsilon$ tolerance deviation between F(x) and the real value. Therefore, the correct prediction dot is in the following closed area

Fig. 10

Soft interval support vector. The red points indicate correct predictions, and the blue points indicate predictions that do not meet the precision requirements. (Color figure online)

The SVR problem is further transformed into

$$\begin{aligned} \underset{w,b}{min}\frac{1}{2}\left\| \omega \right\| ^{2}+C\sum _{i=1}^{m}\iota (f(x_i)-y_i). \end{aligned}$$

(35)

$\iota$ is an insensitive function, and $\omega$ is the corresponding weight matrix. We should pay special attention to the constant C, that is the regularization constant (also known as the penalty factor). The higher C is, the less tolerable the error, which leads to over-fitting. The smaller C is, the less fitting it would be, which leads to under-fitting. A too large or too small value of C can lead to poor generalization ability. Therefore, when using the SVR model, the parameter C is one of the two important parameters. It is often difficult to get the above results directly, and therefore we use the idea of relaxation and approximation to turn the problem into another problem that is easy to handle.

After the relaxation variable is introduced, the upper formula is transformed into an unconstrained minimization problem by introducing the Lagrange multiplier method. Finally, the objective function that needs to be optimized is

$$\begin{aligned} L(\omega ,b,\alpha ,{\hat{\alpha }},\xi ,{\hat{\xi }},\mu ,{\hat{\mu }}) \end{aligned}$$

(36)

$$\begin{aligned} =\frac{1}{2}\left\| \omega \right\| ^{2}+C\sum _{i=1}^{m}(\xi _i+\hat{\xi _i})-\sum _{i=1}^{m}\mu _i \xi _i- \sum _{i=1}^{m}\hat{\mu _i}\hat{\xi _i}+\sum _{i=1}^{m} \alpha _i(f(x_i)-y_i-\epsilon -{\hat{\xi }}_i) \end{aligned}$$

(37)

$$\begin{aligned} +\sum _{i=1}^{m}{\hat{\alpha }}_i(y_i-f(x_i)-\epsilon -{\hat{\xi }}_i). \end{aligned}$$

(38)

We convert the above formula into the SVR dual problem. If the original formula has a solution, it must satisfy the KKT condition. In the end, it can be expressed as follows:

$$\begin{aligned} f(x)=\sum _{i=1}^{m}({\hat{\alpha }}_i-\alpha _i)\kappa (x_i,x_j)+b, \end{aligned}$$

(39)

$$\begin{aligned} \kappa (x_i,y_i)=\varPhi (x_i)^{\top }\varPhi (x_j), \end{aligned}$$

(40)

where $\kappa$ ($\cdot$) is a kernel function. For regression problems, the selection of the kernel function often has great influence on the regression effect. Therefore, multi-core learning is a direction to solve this problem. In this study we can choose a Gaussian kernel or a sigmoid kernel. After a kernel function is set, it will introduce another important parameter, the kernel parameter gamma, which needs to be adjusted. This kernel parameter implicitly determines the distribution situation after attribute variables are mapped to the reproducing Hilbert space (characteristic space). The larger the gamma, the less support vectors there will be. The smaller the gamma, the more support vectors there will be. The number of support vectors affects the speed of training and prediction.

After the kernel function is determined, the final objective function is converted to the following formula by processing the relative equation of the kernel matrix:

$$\begin{aligned} \underset{\omega }{max}J(\omega )=\frac{\omega ^{\top }S_{b}^{\phi }\omega }{\omega ^{\top }S_{w}^{\phi }\omega }. \end{aligned}$$

(41)

This equation is equivalent to the original objective function that needs to be optimized. To date, the theoretical derivation of SVR is the end.

SVR has been successfully applied in text processing because in the process of feature engineering, SVR often uses every single word as an attribute. Through the VC dimension theory, we can conclude that its description ability is sufficient to break up different documents, and so it will show the sparsity in the global arena. This characteristic is in line with the principle of SVR. Considering the similarity of the problem, and according to the dataset analysis of the system, the data distribution shows sparsity. This is the principal support of choosing SVR as the main prediction model in the present study.