ICAT-net: a lightweight neural network with optimized coordinate attention and transformer mechanisms for earthquake detection and phase picking

Abstract

Seismic signal detection is a crucial technology for enhancing the efficiency of earthquake early warning systems. However, existing deep learning-based seismic signal detection models often face limitations in resource-constrained seismic monitoring engineering environments due to the high computational resource demands of the models. To address this issue, this study employs spatial-depth convolution techniques in the downsampling process of seismic signal sequences, effectively minimizing the loss of fine-grained feature information. Concurrently, we leverage the coordinate attention module to enhance the model’s ability to recognize spatial features in seismic signal sequences. To reduce computational costs, we map the keys and values in the transformer architecture to a lower-dimensional subspace, significantly decreasing the demand for computational resources. By employing concatenation operations between the encoder and decoder, the model retains rich contextual information and progressively restores the spatial resolution of the signal during the decoding process. Based on these models, we propose the integration of coordinate attention and transformer network (ICAT-net), an efficient multi-task network designed to simultaneously handle various tasks, including seismic sequence recognition and phase picking. ICAT-net integrates local feature relationships with long-range dependency processing capabilities to meet the requirements of multi-task learning. Experimental results demonstrate that ICAT-net requires only 4.743168G of floating-point operations (FLOPs) and has a parameter count of 0.260755M, while performing excellently in tasks such as seismic waveform detection (DET), P-wave phase picking (Ppk), and S-wave phase picking (Spk). These advantages render ICAT-net particularly suitable for deployment in resource-constrained environments, providing valuable solutions for earthquake monitoring and disaster risk assessment.

Appendices

Appendix A Up sampling block

Figure 8 illustrates the detailed workflow of the upsampling module. Following processing by the ICAT module, the feature matrix initially flows to a one-dimensional transposed convolution layer (Conv Transpose1d). The purpose of this layer is to perform upsampling operations to extend the sequence length of features via transposed convolution, which is utilized for expanding the feature dimensions of one-dimensional signals or time series data. Upon completion of this process, the feature matrix is fed into a one-dimensional convolution layer (Conv 1d), which is equipped with a convolution kernel of variable size K and set to a stride of 1. This layer adapts to the needs of feature extraction at various scales; by selecting an appropriate convolution kernel size K, it effectively integrates contextual information within the sequence, thereby enhancing the model’s ability to represent features at different temporal scales in time series data. In the subsequent process, the data pass through a one-dimensional batch normalization layer (BatchNorm1d), which stabilize the training process by normalizing batch data, improve model performance, and aids in preventing overfitting. Ultimately, the data undergo a nonlinear transformation via the Gaussian error linear unit (GELU) activation function, which captures and enhances the model’s comprehension and representation of complex data patterns, thereby providing a rich feature representation for downstream tasks.

Fig. 8

Upsampling architecture

Appendix B Training details

In this experiment, two NVIDIA GeForce RTX 3090 GPUs and an Intel Core i9-10900X CPU were used for computation, and the experiment ran for a total of 8 hours. We implemented the training of the ICAT-net using PyTorch and applied a series of meticulously designed initialization and optimization techniques to ensure that the model effectively learns and captures the complex features of seismic wave signals. During training, we configured the parameters based on the settings used in the SeisT [25] method. Early stages of model training are critical for ensuring stable and effective convergence. Proper initialization and careful monitoring during these stages help prevent issues such as vanishing or exploding gradients, facilitating a smooth learning process [47]. This initialization method can prevent issues of gradient vanishing or exploding, helping to stabilize weight updates in the early stages of training. Additionally, the weights of the BatchNorm layers were set to 1, while biases were initialized to 0. This configuration helps the model to have nonzero gradients at the start of training, facilitating the onset of the learning process. To effectively train the ICAT-net, we used binary cross-entropy (BCE) as the loss function.

In terms of optimization strategies, we selected the Adam optimizer [48], an adaptive learning rate optimization algorithm that adjusts the learning rate for each parameter based on the history of parameter updates. This allows the model to learn at different step sizes in different parameter space regions, which is crucial for deep networks. Combined with the cyclic learning rate scheduler [49], our learning rate progressively decreases from $1 \times 10^{-3}$ to $8 \times 10^{-5}$ over a cycle, then repeats. The periodic adjustment of the learning rate aims to balance the model’s exploration (large step updates to parameters to escape local minima or saddle points) and exploitation (small step updates to refine the current solution). To prevent overfitting and ensure the generalizability of the model, we implemented an early stopping strategy. Specifically, if no decrease in loss on the validation set is observed over 20 consecutive training epochs, the training process will be prematurely terminated. This strategy ensures that the model does not waste resources on ineffective learning while preserving the state of the model at its current best performance.

Based on the EQTransformer [36] configuration, this study sets the detection thresholds for P-waves (primary longitudinal waves) and S-waves (secondary transverse waves) at 0.3, while the threshold for detecting seismic events is set at 0.5. These thresholds are meticulously chosen to balance detection rates and false alarm rates, ensuring that the model maintains high sensitivity while accurately excluding non-seismic signals. In the probability distribution of the model’s output, the arrival time of a phase is determined by locating the peak of the distribution. Specifically, if the output probability exceeds the corresponding threshold, the model marks the respective time point as a potential seismic phase arrival.

Appendix C Evaluation

Different evaluation metrics are crucial for assessing a model’s performance in specific tasks. This study utilizes multiple statistical metrics to comprehensively evaluate the model’s performance, including precision (${\text {Pr}}$) [50], recall (${\text {Re}}$) [50], F1 score (F1) [51], mean absolute error (${\text {MAE}}$) [52], standard deviation (${\text {Std}}$) [53], and mean error (${\text {Mean}}$) [53], defined as follows:

$$\begin{aligned} {\text {Pr}} = \frac{{T_{{\text {p}}} }}{{F_{{\text {p}}} + T_{{\text {p}}} }} \end{aligned}$$

(C1)

$$\begin{aligned} {\text {Re}} = \frac{{T_{{\text {p}}} }}{{F_{{\text {n}}} + T_{{\text {p}}} }} \end{aligned}$$

(C2)

$$\begin{aligned} F1 = \frac{{2 \times {\text {Pr}} \times {\text {Re}}}}{{{\text {Pr}} + {\text {Re}}}} \end{aligned}$$

(C3)

$$\begin{aligned} \text {Mean}= & \frac{1}{N} \sum _{i=1}^{N} (y_i - {\hat{y}}_i) \end{aligned}$$

(C4)

$$\begin{aligned} \text {Std}= & \sqrt{\frac{1}{N} \sum _{i=1}^{N} (y_i - {\hat{y}}_i)^2} \end{aligned}$$

(C5)

$$\begin{aligned} \text {MAE}= & \frac{1}{N} \sum _{i=1}^{N} |y_i - {\hat{y}}_i| \end{aligned}$$

(C6)

Specifically, the definitions of these evaluation metrics are as follows: $T_{{\text {p}}}$ represents the number of true positives, $F_{{\text {p}}}$ represents the number of false positives, $F_{{\text {n}}}$ represents the number of false negatives, N is the total number of samples, $y_i$ represents the true label of sample i, and ${\hat{y}}_i$ represents the predicted value for sample i. In phase picking tasks, this study considers samples with residuals within an error range of $\delta < 0.1\,s$ as true positives, in order to accurately assess model performance. This criterion helps identify samples with smaller residuals and reduces the number of false positives.