An improved extreme learning machine algorithm for transient electromagnetic nonlinear inversion

Highlight

• The F-ELM is applied for TEM inversion.

• The F-ELM is proposed to eliminate the multicollinearity problem.

• The fractal dimension technology greatly enhances the training speed of the ELM algorithm without losing the main statistical information.

Abstract

Transient electromagnetic method (TEM) inversion is significantly nonlinear. To eliminate the multicollinearity problem faced by the extreme learning machine (ELM) algorithm for TEM inversion, an improved ELM algorithm (F-ELM) based on fractal dimension technology is proposed. By reducing the dimension of the hidden layer output matrix (H) based on fractal dimension theory without losing the main statistical information, the proposed algorithm can not only guarantee the full column rank of the newly produced hidden layer output matrix $\left(H^{\prime }\right)$ but also enhance the training speed of the overall process. To prove the effectiveness of the F-ELM algorithm, a synthetic example and a field example using TEM inversion are established in this study. The experimental results illustrate that compared with the ordinary ELM algorithm and its variants, the proposed algorithm greatly reduces the computing time, improves the inversion accuracy and stability of the algorithm. Furthermore, it is also proven that the F-ELM algorithm is a very effective technique for TEM inversion.

Introduction

The extreme learning machine (ELM) is a novel non-iterative machine learning algorithm, which is a single-hidden-layer feedforward neural network (SLFN) presented by Huang et al. (2006). Compared with other well-known neural networks (e.g., RBF, RNN and LSTM), the ELM algorithm contains the model parameters (i.e., the input-layer weights and the hidden-layer thresholds) that are produced randomly without the need for manual adjustment, and the output-layer weights are uniquely obtained via the analytic solution that is based on the least squares method (Singh et al., 2021; Ragusa et al., 2020). Unlike gradient-based neural network learning approaches, the ELM achieves a fast training speed and eliminates the possibility of convergence to local minima. Currently, the ELM algorithm is widely applied in regression and classification (Cai et al., 2019; She et al., 2018), image recognition (Cheng et al., 2019; Song et al., 2020) and decision support (Li et al., 2019a) due to its good generalization ability.

Thus far, the ELM algorithm has attracted increasing attention. But the multicollinearity in the ELM algorithm worsens its generalization ability for TEM inversion. On the one hand, to eliminate the multicollinearity problem by improving the output weight of the ELM, Huang added a threshold to each matrix element when calculating the output matrix (Huang et al., 2006); this approach connects ELM theory with other mathematical theories (e.g., ridge regression and matrix theory). In addition, some scholars have utilized LDL^T to decompose the output matrix and eliminate the multicollinearity problem (Zhang et al., 2014). On the other hand, the multicollinearity problem can also be solved by selecting the proper hidden layer node parameters. Huang et al. (Feng et al., 2009a; Huang et al., 2008) defined the incremental extreme learning machine (I-ELM), where the hidden layer nodes are incrementally added until the error satisfies a predetermined value. Miche et al. (Adnan et al., 2019) proposed an optimal-pruning extreme learning machine (OP-ELM), which initializes the hidden-layer thresholds randomly and sorts the features obtained by conducting hidden layer node transformation for the training set. To date, ELM algorithms have been used to try to solve the multilinearity problem through improving the output-layer weights, as well as avoiding the random selection of initial hidden layer nodes. Inspired by these latest methods, we propose a modified ELM (F-ELM) algorithm based on fractal dimension technology to eliminate the multicollinearity problem. The proposed algorithm utilizes fractal dimension technology to retrieve the information of the hidden layer output matrix for the purpose of estimating its node parameters. Compared with the ordinary ELM and its variant methods, the presented F-ELM approach can guarantee that the unbiased estimation of the output weight matrix. In addition, the proposed F-ELM algorithm can also greatly reduce the computing cost and enhance the accuracy of the entire process.

The transient electromagnetic method (TEM) is currently one of the most popular non-seismic exploration methods and is widely applied in resource exploration (groundwater, minerals, oil and gas, etc.) and shallow geological surveys (Yu et al., 2018; Li et al., 2016). TEM inversion is a typical technology with nonlinear features used to solve the geoelectric model parameters (resistivity and thickness). However, the inversion problem is usually ill-posed and nonunique, and the uncertainty of the inversion experiment greatly aggravates the stability of the model parameters. Therefore, the TEM data inversion algorithm directly affects its applicability.

In the past few decades, various linear or quasi-linear TEM inversion algorithms have been published (Loke and Barker, 1995, 1996). But these traditional algorithms have some common shortcomings: they mostly deal with nonlinear problems using linear approximation strategies, and they rely heavily on the initially selected model (El-Qady and Ushijima, 2001). In the latest years, machine learning methods have been widely applied in the field of geophysics, especially artificial neural networks (ANNs) (Guo et al., 2021; Jorge et al., 2021; Hadrien et al., 2020; Singh et al., 2013; Mandolesi et al., 2018; Li et al., 2020a). ANNs have been developed that do not require complex forward model calculations in the iterative process; thus, they have been widely employed in TEM inversion interpretations with high computational efficiency and strong learning capabilities (Noh et al., 2019; Li et al., 2019b; Puzyrev and Swidinsky, 2020; Bai et al., 2020; Asif et al., 2021). However, these gradient-based algorithms have a common defect: them typically have difficulty converging and easily fall into local optima during training (Chandwani et al., 2015). The ELM algorithm uses the Moore-Penrose generalized inverse to solve the calculation problem during training, which overcomes the inherent shortcoming of the gradient-based learning algorithms (Huang et al., 2006). In this study, a layered geoelectric model based on TEM inversion with high-dimensional and nonlinear features is established, and then the F-ELM algorithm is proposed for reducing the dimension of the output weight matrix in the ELM that is used for TEM inversion. Finally, a reliable and efficient ELM network model for TEM inversion is obtained.

In this study, a modified ELM algorithm named F-ELM is presented based on fractal dimension technology. Compared with previous modified ELM algorithms, the proposed F-ELM algorithm can deal with multicollinearity problems while keeping the output weight estimation unbiased. The F-ELM algorithm achieves strong performance when dealing with data containing interference noise generated by the TEM exploration. In order to evaluate the effectiveness of the F-ELM in TEM inversion applications, a layered geoelectric model subject to the TEM inversion algorithm is built, and a synthetic example and a field example are used for experimental simulation. The results indicate that the F-ELM algorithm achieves better computational efficiency, better generalization ability and stronger stability than other famous variants of the ELM. It is worth mentioning that fractal dimension theory is also applied to the ELM method in the literature (Tanyildizi, 2015). However, in this study, fractal dimension technology is adopted to handle the output weight matrix instead of the original sampled data as in (Tanyildizi, 2015).

The remainder of this paper is arranged as follows: Section 2 shows the main ideas of the F-ELM approach. Section 3 introduces the basic theory of the TEM method. The simulation results are given in Section 4. An example application is demonstrated in Section 5. Discussions and conclusions are presented in Sections 6 Discussion, 7 Conclusion, respectively.