BERT-based chinese text classification for emergency management with a novel loss function

Abstract

This paper proposes an automatic Chinese text categorization method for solving the emergency event report classification problem. Since the bidirectional encoder representations from transformers (BERT) has achieved great success in the natural language processing domain, it is employed to derive emergency text features in this study. To overcome the data imbalance problem in the distribution of emergency event categories, a novel loss function is proposed to improve the performance of the BERT-based model. Meanwhile, in order to avoid the negative impacts of the extreme learning rate, the Adabound optimization algorithm that achieves a gradual smooth transition from Adam optimizer to stochastic gradient descent optimizer is employed to learn the parameters of the model. The feasibility and competitiveness of the proposed method are validated on both imbalanced and balanced datasets. Furthermore, the generic BERT, BERT ensemble LSTM-BERT (BERT-LB), Attention-based BiLSTM fused CNN with gating mechanism (ABLG-CNN), TextRCNN, Att-BLSTM, and DPCNN are used as benchmarks on these two datasets. Meanwhile, sampling methods, including random sampling, ADASYN, synthetic minority over-sampling techniques (SMOTE), and Borderline-SMOTE, are employed to verify the performance of the proposed loss function on the imbalance dataset. Compared with benchmarking methods, the proposed method has achieved the best performance in terms of accuracy, weighted average precision, weighted average recall, and weighted average F1 values. Therefore, it is promising to employ the proposed method for real applications in smart emergency management systems.