Abstract
The goal of cross-domain named entity recognition is to transfer models learned from labelled source domain data to unlabelled or lightly labelled target domain datasets. This paper discusses how to adapt a cross-domain sentiment analysis model to the field of named entity recognition, as the sentiment analysis model is more relevant to the tasks and data characteristics of named entity recognition. Most previous classification methods were based on a tokenwise approach, and this paper introduces entity boundary information to prevent the model from being affected by a large number of nonentity labels. Specifically, adversarial training is used to enable the model to learn domain-confusing knowledge, and contrastive learning is used to reduce domain shift problems. The entity boundary information is transformed into a global boundary matrix representing sentence-level target labels, enabling the model to learn explicit span boundary information. Experimental results demonstrate that this method achieves good performance compared to multiple cross-domain named entity recognition models on the SciTech dataset. Ablation experiments reveal that the method of introducing entity boundary information significantly improves KL divergence and contrastive learning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ganin, Y., Ustinova, E., Ajakan, H., et al.: Domain-adversarial training of neural networks. J. Machine Learn. Res. 17(1), 2096–2030 (2016)
Zhuang, F., Cheng, X., Luo, P., et al.: Supervised representation learning: Transfer learning with deep autoencoders. In: Twenty-Fourth International Joint Conference n Artificial Intelligence (2015)
Shah, D.J., Lei, T., Moschitti, A., et al.: Adversarial domain adaptation for duplicate question detection. arXiv preprint arXiv:1809.02255 (2018)
Du, C., Sun, H., Wang, J., et al.: Adversarial and domain-aware BERT for cross-domain sentiment analysis. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4019–4028 (2020)
Karouzos, C., Paraskevopoulos, G., Potamianos, A.: UDALM: Unsupervised domain adaptation through language modeling. arXiv preprint arXiv:2104.07078. (2021)
Saito, K., Ushiku, Y., Harada, T.: Asymmetric tri-training for unsupervised domain adaptation. In: International Conference on Machine Learning. PMLR, pp. 2988–2997 (2017)
Lee, S., Kim, D., Kim, N., et al.: Drop to adapt: Learning discriminative features for unsupervised domain adaptation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 91–100 (2019)
Chen, T., Kornblith. S., Norouzi, M., et al.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning. PMLR, pp. 1597–1607 (2020)
Khosla, P., Teterwak, P., Wang, C., et al.: Supervised contrastive learning. Adv. Neural. Inf. Process. Syst. 33, 18661–18673 (2020)
Rui, W., et al.: Crossdomain contrastive learning for unsupervised domain adaptation. ArXiv preprint, arXiv:2106.05528 (2021)
Park, C., Lee, J., Yoo, J., et al.: Joint contrastive learning for unsupervised domain adaptation. arXiv preprint arXiv:2006.10297 (2020)
Qu, Y., Shen, D., Shen, Y., et al.: Coda: Contrast-enhanced and diversity-promoting data augmentation for natural language understanding. arXiv preprint arXiv:2010.08670 (2020)
Gao, T., Yao, X., Chen, D.: Simcse: Simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821 (2021)
Long, Q., Luo, T., Wang, W., Pan, S.J.: Domain Confused Contrastive Learning for Unsupervised Domain Adaptation. arXiv:2207.04564v1 [cs.CL] (2022)
Strubell, E., Verga, P., Belanger, D., et al.: Fast and accurate entity recognition with iterated dilated convolutions. arXiv preprint arXiv:1702.02098 (2017)
Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991, (2015)
Chiu, J.P.C., Nichols, E.: Named entity recognition with bidirectional LSTM-CNNs. Trans. Assoc. Comput. Linguist. 4, 357–370 (2016)
Zhang, Y., Yang, J.: Chinese NER using lattice LSTM. arXiv preprint arXiv:1805.02023 (2018)
Cao, P., Chen, Y., Liu, K., et al.: Adversarial transfer learning for Chinese named entity recognition with self-attention mechanism. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp.182–192 (2018)
Zhou, J.T., Zhang, H., Jin, D, et al.: Dual adversarial neural transfer for low-resource named entity recognition. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3461–3471 (2019)
Chen, L., Moschitti, A.: Transfer learning for sequence labeling using source model and target data. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33(01), pp. 6260–6267 (2019)
Tong, M., Wang, S., Xu, B., et al.: Learning from miscellaneous other-class words for few-shot named entity recognition. arXiv preprint arXiv:2106.15167 (2021)
Su, J., Lu, Y., Pan, S, et al.: Roformer: Enhanced transformer with rotary position embedding. arXiv preprint arXiv:2104.09864 (2021)
Sang, E.F., De Meulder, F.: Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. arXiv preprint cs/0306050 (2003)
Jia, C., Liang, X., Zhang, Y.: Cross-domain NER using cross-domain language modeling. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 2464–2474 (2019)
Madry, A., Makelov, A., Schmidt, L., et al.: Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017)
Lample, G., Ballesteros, M., Subramanian, S., et al.: Neural architectures for named entity recognition. arXiv preprint arXiv:1603.01360 (2016)
Jia, C., Zhang, Y.: Multi-cell compositional LSTM for NER domain adaptation. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5906–5917 (2020)
Peng, Q., Zheng, C., Cai, Y., et al.: Unsupervised cross-domain named entity recognition using entity-aware adversarial training. Neural Netw. 138, 68–77 (2021)
Acknowledgements
This work was supported by the National Key R&D Program of China under Grant No. 2020YFB1710200.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wang, Y., Shi, C., Li, L., Guo, M. (2023). GPDCCL: Cross-Domain Named Entity Recognition with Span-Based Domain Confusion Contrastive Learning. In: Yu, Z., et al. Data Science. ICPCSEE 2023. Communications in Computer and Information Science, vol 1880. Springer, Singapore. https://doi.org/10.1007/978-981-99-5971-6_15
Download citation
DOI: https://doi.org/10.1007/978-981-99-5971-6_15
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-5970-9
Online ISBN: 978-981-99-5971-6
eBook Packages: Computer ScienceComputer Science (R0)