Research on enterprise text classification methods of BiLSTM and CNN based on BERT

The traditional enterprise text data classification method ignores the context of the text. Each word is independent from each other and cannot represent semantic information. The text description and classification effect is poor, and the feature engineering needs human intervention, so the generalization ability is not strong. In view of the low efficiency and accuracy of enterprise text data classification, this paper proposes a bidirectional encoder representation based on Transformer (BERT) The enterprise text classification model BBLC-ATT based on convolutional neural networks (CNN) and bi-directional long short-term memory (BiLSTM) neural networks and attention mechanism (Attention). The model uses BERT training word vector and combines the features of CNN and BiLSTM to capture local potential features and context information. Secondly, the feature vectors extracted from the hybrid network layer are input into the self-attention layer to extract the syntactic and semantic features between words in the enterprise text sentences. Finally, this paper compares BBLC-ATT model with traditional deep learning model in terms of accuracy, accuracy, recall and F1 value. The experimental results show that the BBLC-ATT model is superior to other models in all evaluation indicators, and the accuracy rate is increased by 3.28% - 15.86%.