Abstract
Named entity recognition (NER) has always been an important research task in information extraction and knowledge graph construction. Due to the randomness of Chinese user-generated reviews, character substitution and informal expression are very common. Its widespread phenomenon leads to that Chinese car reviews NER is still a major challenge. In this paper, we propose a joint multi-view character embedding model for Chinese NER (JMCE-CNER) of car reviews. Firstly, deeper character features are extracted from pronunciation, radical, and glyph views to generate the multi-view character embedding. Secondly, a car domain dictionary is constructed for providing accurate word-level information. Thirdly, the multi-view character embedding and the word-level embedding are jointly fed into the deep learning model to perform the Chinese car reviews NER. The experimental datasets of Chinese car reviews are obtained by manual annotation, containing four types of entities, namely brand, model, attribute and structure of the car. The experimental results on the Chinese car review datasets demonstrate that our proposed model achieves the optimal performance compared with the other state-of-the-art models. Furthermore, the model substantially reduces the impact of character substitution and informal expression on performing NER tasks.
Similar content being viewed by others
Data availability
The data that support the findings of this study are available from the corresponding author, Anning Wang, upon reasonable request.
References
Xu X, Wang X, Li Y, Haghighi M (2017) Business intelligence in online customer textual reviews: understanding consumer perceptions and influential factors. Int J Inf Manage 37(6):673–683
Ravi K, Ravi V (2015) A survey on opinion mining and sentiment analysis: tasks, approaches and applications. Knowl-Based Syst 89:14–46
Liu Z, Qin C, Zhang Y (2020) Mining product competitiveness by fusing multisource online information. Decis Support Syst 143(5):113477
Qi J, Zhang Z, Jeon S, Zhou Y (2016) Mining customer requirement from online reviews: a product improvement perspective. Soc Sci Electron Publ 53:951–963
Goyal A, Gupta V, Kumar M (2018) Recent named entity recognition and classification techniques: a systematic review. Comput Sci Rev 29:21–43
Li D, Yan L, Yang J, Ma Z (2022) Dependency syntax guided bert-bilstm-gam-crf for chinese ner. Expert Syst Appl 196:116682
Sharma R, Morwal S, Agarwal B, Chandra R, Khan MS (2020) A deep neural network-based model for named entity recognition for hindi language. Neural Comput Appl 32(20):16191–16203
Derczynski L, Maynard D, Rizzo G, Erp MV, Gorrell G, Troncy R, Petrak J, Bontcheva K (2015) Analysis of named entity recognition and linking for tweets. Inf Process Manage 51:32–49
Kainan J, Xin L, Rongchen Z (2021) Overview of chinese domain named entity recognition. Comput Eng Appl 57:1–15
Wang Y, Lu L, Wu Y, Chen Y (2022) Polymorphic graph attention network for chinese ner. Expert Syst Appl, 117467
Peng DL, Wang YR, Liu C, Chen Z (2020) TL-NER: a transfer learning model for chinese named entity recognition. Inf Syst Front 22(1):1291–1304
Khalifa M, Shaalan K (2019) Character convolutions for arabic named entity recognition with long short-term memory networks. Comput Speech Lang 58:335–346
Gui T, Ma R, Zhang Q, Zhao L, Huang X (2019) Cnn-based chinese ner with lexicon rethinking. In: Twenty-eighth international joint conference on artificial intelligence IJCAI-19, pp 4982–4988
Shan Z, Rui L, Zhiping C (2022) Survey of chinese named entity recognition. J Front Comput Sci Technol 16(2):296
Zhang N, Li F, Xu G, Zhang W, Yu H (2019) Chinese ner using dynamic meta-embeddings. IEEE Access 92:103133
Wang Q et al (2019) Incorporating dictionaries into deep neural networks for the chinese clinical named entity recognition. J Biomed Inform 92:103133
Fang Z, Qiang Z et al (2021) Referent graph embedding model for name entity recognition of chinese car reviews. Knowl Based Syst 233:107558
Asgari-Chenaghlu M, Feizi-Derakhshi MR, Farzinvash L, Balafar M, Motamed C (2022) Cwi: a multimodal deep learning approach for named entity recognition from social media using character, word and image features. Neural Comput Appl 34(3):1905–1922
Li Y, Du G, Xiang Y, Li S, Chen H (2020) Towards chinese clinical named entity recognition by dynamic embedding using domain-specific knowledge. J Biomed Inform 106:103435
Gaio M, Moncla L (2017) Extended named entity recognition using finite-state transducers: An application to place names. In: International conference on advanced geographic information systems, applications, and services, pp 15–20
Ling L et al (2018) An attention-based bilstm-crf approach to document-level chemical named entity recognition. Bioinformatics 34(8):1381–1388
Li J, Meng K (2021) MFE-NER: Multi-feature fusion embedding for chinese named entity recognition. arXiv preprint arXiv:2109.07877
Liu Z, Zhu C, Zhao T (2010) Chinese named entity recognition with a sequence labeling approach: Based on characters, or based on words? In: International conference on advanced intelligent computing theories & applications, pp 634–640
Zhao J, Xie X, Xu X, Sun S (2017) Multi-view learning overview: recent progress and new challenges. Inform Fus 38:43–54
Jia X, Jing XY, Zhu X, Cai Z, Hu CH (2021) Co-embedding: a semi-supervised multi-view representation learning approach. Neural Comput Appl 34(6):4437–4457
Ding Z, Shao M, Fu Y (2018) Robust multi-view representation: A unified perspective from multi-view learning to domain adaption. In: Twenty-seventh international joint conference on artificial intelligence IJCAI-18, pp 5434–5440
Guo Q, Guo Y (2022) Lexicon enhanced chinese named entity recognition with pointer network. Neural Comput Appl 34:14535–14555
Xiaofeng M, Wei W, Aiping X (2020) Incorporating token-level dictionary feature into neural model for named entity recognition. Neurocomputing 375:43–50
Nie Y, Zhang Y, Peng Y, Yang L (2022) Borrowing wisdom from world: modeling rich external knowledge for chinese named entity recognition. Neural Comput Appl 34(6):4905–4922
Hkiri AOE, Mallat S, Zrigui M (2016) Improving coverage of rule based ner systems. In: International conference on information & communication technology & accessibility (ICTA), pp 1–6
Gerner M, Nenadic G, Bergman CM (2010) Linnaeus: a species name identification system for biomedical literature. BMC Bioinformatics 11(1):85
Pande SD, Kanna RK, Qureshi I et al (2022) Natural language processing based on name entity with n-gram classifier machine learning process through ge-based hidden markov model. Mach Learn Appl Eng Educ Manag 2(1):30–39
Patil N, Patil A, Pawar B (2020) Named entity recognition using conditional random fields. Proced Comput Sci 167:1181–1188
Tarasova O, Rudik A, Biziukova NY, Filimonov D, Poroikov V (2022) Chemical named entity recognition in the texts of scientific publications using the naïve bayes classifier approach. J Chemin 14(1):1–12
Morwal S (2012) Named entity recognition using hidden markov model (HMM). Int J Comput Vision 1(4):15–23
Teixeira J, Sarmento L, Oliveira EC (2011) A bootstrapping approach for training a ner with conditional random fields. In: Progress in artificial intelligence,15th Portuguese conference on artificial intelligence, pp 664–678
Mcdonald R, Pereira F (2005) Identifying gene and protein mentions in text using conditional random fields. BMC Bioinformatics 6:6
Wang J, Lin C, Li M, Zaniolo C (2020) Boosting approximate dictionary-based entity extraction with synonyms. Inf Sci 530(1):1–21
Tran VC, Nguyen NT, Fujita H, Hoang DT, Hwang D (2017) A combination of active learning and self-learning for named entity recognition on twitter using conditional random fields. Knowl Based Syst 132(15):179–187
Shen Y, Yun H, Lipton Z, Kronrod Y, Anandkumar A (2017) Deep active learning for named entity recognition. In: Proceedings of the 2nd workshop on representation learning for NLP, pp 252–256
Lin Y, Hong L, Yi L, Li X, Anwar MW (2015) Biomedical named entity recognition based on deep neutral network. Int J Hybrid Inform Technol 8(8):279–288
Li P, Dong R, Wang Y, Chou J, Ma W (2017)Leveraging linguistic structures for named entity recognition with bidirectional recursive neural networks. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 2664–2669
Gridach M (2017) Character-level neural network for biomedical named entity recognition. J Biomed Inform 70:85–91
Devlin J, Chang M.W, Lee K, Toutanova K (2018) BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805
Peters M, Neumann M, Iyyer M, Gardner M, Zettlemoyer L (2018) Deep contextualized word representations. arXiv preprint arXiv:1802.05365
Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. Adv Neural Inform Process Syst 26
Li X, Zhang H, Zhou XH (2020) Chinese clinical named entity recognition with variant neural structures based on BERT methods. J Biomed Inform 107(5):103422
Wang J, Xu W, Fu X, Xu G, Wu Y (2020) ASTRAL: adversarial trained LSTM-CNN for named entity recognition. Knowl Based Syst 197:105842
Akbik A, Blythe D, Vollgraf R (2018) Contextual string embeddings for sequence labeling. In: Proceedings of the 27th International conference on computational linguistics, pp 1638–1649
Huang Z, Xu W, Yu K (2015) Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991
Ma X, Hovy E (2016) End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of the 54th annual meeting of the association for computational linguistics, vol 1, pp 1064–1074
Zhang Y, Yang J (2018) Chinese NER using lattice LSTM. arXiv preprint arXiv:1805.02023
Li S.Y, Jiang Y, Zhou Z.H (2014) Partial multi-view clustering. In Proceedings of the AAAI conference on artificial intelligence, pp 1968–1974
Liu J, Gao L, Guo S, Ding R, Thiruvady D (2021) A hybrid deep-learning approach for complex biochemical named entity recognition. Knowl Based Syst 221:106958
Lee L, Lu Y (2021) Multiple embeddings enhanced multi-graph neural networks for chinese healthcare named entity recognition. IEEE J Biomed Health Inform 25(7):2801–2810
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A.N, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inform Process Syst 30
Lafferty J, Mccallum A, Pereira F (2001) Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Processing 18th international conference on machine learning, pp 282–289
Forney GD (1993) The viterbi algorithm. In: Proceedings of the IEEE, vol 61, pp 268–278
Dong C, Zhang J, Zong C, Hattori M, Hui D (2016) Character-based lstm-crf with radical-level features for chinese named entity recognition. In: Natural language understanding and intelligent applications, pp 239–250
Acknowledgements
This work was supported by grants from the National Natural Science Foundation of China (Nos. 72101078, 72071060, 72101075, 72201087, 72171069, and 72188101) and the Fundamental Research Funds for the Central Universities (NOs. JZ2021HGTA0131 and JZ2022HGTB0286).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix A Analysis of time complexity
Appendix A Analysis of time complexity
First, we analyze our model’s time complexities. Then, the time complexity of our model is compared to the state-of-the-art models described in the main body.
Some important variables are defined as follows: l: the maximum text length; \({d_1}\): character embedding size; \({d_2}\): pronunciation/radical/glyph/word embedding size; \({d_3}\): hidden layer dimension of LSTM; \(\lambda\): the number of hidden layers (BERT); m: the number of labels; N: the total data amount; t: the data amount of a batch; n: the number of iterations; g: the maximum number of glyphs of the Chinese characters.
1.1 A. 1 Time complexity of the JMCE-CNER model
In this section, we analyze the time complexity of the components in the JMCE-CNER model, which is divided into four parts: embedding presentation layer, Bi-LSTM layer, attention mechanism layer, and CRF layer.
Embedding presentation layer. First, the character embedding is generated by BERT pre-trained model, whose time complexity is \(O\left( {\lambda {l^2}{d_1} + \lambda ld_1^2} \right)\); second, the pronunciation/radical/word embedding is generated by the word2vec model, whose time complexities are \(O\left( {ld_2^2} \right)\); Third, the glyph embedding is generated by the word2vec model and Bi-LSTM layer, whose time complexity is \(O\left( {gld_2^2} \right)\). Therefore, the time complexity of embedding presentation layer is \(O\left( {\lambda {l^2}{d_1} + \lambda ld_1^2 + gld_2^2} \right)\).
Bi-LSTM layer. First, the time complexity of each LSTM cell is \(O\left( {d_3^2 + \left( {{d_1} + 4{d_2}} \right) {d_3}} \right)\); second, the time complexity of calculating LSTM output is \(O\left( {{d_3}m} \right)\). Therefore, the time complexity of Bi-LSTM layer is \(O\left( {ld_3^2 + l\left( {{d_1} + 4{d_2}} \right) {d_3} + l{d_3}m} \right)\).
Attention mechanism layer. This layer consists of three steps: similarity calculation, softmax calculation and weighted summation. First, the time complexity of similarity calculation is \(O\left( {{l^2}{d_3}} \right)\); second, the time complexity of softmax calculation is \(O\left( {{l^2}} \right)\); Third, the time complexity of weighted summation is also \(O\left( {{l^2}{d_3}} \right)\). Therefore, the time complexity of attention mechanism layer is \(O\left( {{l^2}{d_3}} \right)\).
CRF layer. During the training stage, the CRF layer needs to calculate \(\sum \nolimits _{{\tilde{y}} \in {Y_x}} {{e^{Score\left( {S,{\tilde{y}}} \right) }}}\), where \({Y_x}\) is all possible tag sequence (\({m^l}\) in total). By using dynamic programming algorithm, its time complexity can be reduced to \(O\left( {l{m^2}} \right)\). In the inference stage, the Viterbi algorithm is used to find the optimal path, and the time complexity is also \(O\left( {l{m^2}} \right)\). Therefore, the time complexity of CRF layer is \(O\left( {l{m^2}} \right)\).
Overall. Considering all the above time complexity, the time complexity of the JMCE-CNER model can be represented as follows:
1.2 A.2 Comparison with the state-of-the-art models
First, we give the time complexity of each state-of-the-art model. Then, the differences of time complexity between our model and the state-of-the-art models are analyzed.
1.2.1 A.2.1 Time complexities of the state-of-the-art models
The time complexity of each state-of-the-art model can be expressed as:
where u is the number of convolution kernels, and r is the maximum number of radicals of the Chinese character.
Therefore, the size ordering of time complexity is shown as:
1.2.2 A.2.2 Comparative analysis
As shown in Eq. (A.6), the time complexity of the JMCE-CNER model is the second-highest, trailing only the ME-MGNN model. The time complexity of the ME-MGNN model is increased due to the employment of an adapted gated graph sequence neural network (GGSNN) in it. Compared with the CR-CNER model, the BERT pre-trained model is used by the JMCE-CNER, MFE-NER, and BBMC models and the increased time complexity of these models is \(O\left( {nNl\left( {\lambda l{d_1} + \lambda d_1^2} \right) } \right)\). The use of the BERT pre-trained model will inevitably increase the time complexity. The BERT is essentially a tool for generating more accurate character vectors; therefore, these models outperform the CR-CNER model. Furthermore, the JMCE-CNER model has a higher time complexity than the MFE-NER and BBMC models, due to more features of the characters are considered. The BBMC model considers only character features and the MFE-NER model considers character, glyph, and phonetic features. However, the JMCE-CNER model extracts deep character information and enhanced semantic information by using multiple embedding methods to fuse character, pronunciation, radical, and glyph features. Therefore, the performance of the JMCE-CNER model is better than those of the state-of-the-art models but increasing the time complexity.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ding, J., Xu, W., Wang, A. et al. Joint multi-view character embedding model for named entity recognition of Chinese car reviews. Neural Comput & Applic 35, 14947–14962 (2023). https://doi.org/10.1007/s00521-023-08476-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-023-08476-2