Skip to main content

Improving Cross-Domain Named Entity Recognition from the Perspective of Representation

  • Conference paper
  • First Online:
Database Systems for Advanced Applications (DASFAA 2023)

Abstract

Recently, cross-domain named entity recognition (cross-domain NER), which can reduce the high data annotation costs faced by fully-supervised methods, has drawn attention. Most competitive approaches mainly rely on pre-trained language models like BERT to represent words. As such, the original chaotic representations may bring challenges (e.g.,  entity span detection errors and entity type misclassification) for them. Motivated by this, this proposal proposes to improve cross-domain NER by refining the original representations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Here entity types are pre-defined, such as location, organization, and so on.

References

  1. Barros, P., Sciutti, A.: All by myself: learning individualized competitive behavior with a contrastive reinforcement learning optimization. Neural Netw. 150, 364–376 (2022)

    Article  Google Scholar 

  2. Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised learning of visual features by contrasting cluster assignments. NIPS 33, 9912–9924 (2020)

    Google Scholar 

  3. Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning, pp. 1597–1607 (2020)

    Google Scholar 

  4. Chen, X., He, K.: Exploring simple Siamese representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15750–15758 (2021)

    Google Scholar 

  5. Choi, S., Jeong, M., Han, H., Hwang, S.W.: C2l: causally contrastive learning for robust text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 10526–10534 (2022)

    Google Scholar 

  6. Giorgi, J., Nitski, O., Wang, B., Bader, G.: Declutr: deep contrastive learning for unsupervised textual representations. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pp. 879–895 (2021)

    Google Scholar 

  7. Grill, J.B., et al.: Bootstrap your own latent-a new approach to self-supervised learning. Adv. Neural. Inf. Process. Syst. 33, 21271–21284 (2020)

    Google Scholar 

  8. He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729–9738 (2020)

    Google Scholar 

  9. Houlsby, N., et al.: Parameter-efficient transfer learning for NLP. In: International Conference on Machine Learning, pp. 2790–2799 (2019)

    Google Scholar 

  10. Huang, Q., et al.: Entity guided question generation with contextual structure and sequence information capturing. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 13064–13072 (2021)

    Google Scholar 

  11. Jia, C., Zhang, Y.: Multi-cell compositional LSTM for NER domain adaptation. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5906–5917 (2020)

    Google Scholar 

  12. Jiang, D., Ren, H., Cai, Y., Xu, J., Liu, Y., Leung, H.F.: Candidate region aware nested named entity recognition. Neural Netw. 142, 340–350 (2021)

    Article  Google Scholar 

  13. Kenton, J.D.M.W.C., Toutanova, L.K.: Bert: pre-training of deep bidirectional transformers for language understanding. In: NAACL, pp. 4171–4186 (2019)

    Google Scholar 

  14. Lei, X., Cai, Y., Xu, J., Ren, D., Li, Q., Leung, H.: Incorporating task-oriented representation in text classification. In: Li, G., Yang, J., Gama, J., Natwichai, J., Tong, Y. (eds.) DASFAA 2019. LNCS, vol. 11447, pp. 401–415. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-18579-4_24

    Chapter  Google Scholar 

  15. Lin, B.Y., Lu, W.: Neural adaptation layers for cross-domain named entity recognition. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2012–2022 (2018)

    Google Scholar 

  16. Liu, Z., et al.: CrossNER: evaluating cross-domain named entity recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 13452–13460 (2021)

    Google Scholar 

  17. Radford, A., et al.: Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning, pp. 8748–8763 (2021)

    Google Scholar 

  18. Ren, D., Cai, Y., Lei, X., Xu, J., Li, Q., Leung, H.F.: A multi-encoder neural conversation model. Neurocomputing 358, 344–354 (2019)

    Article  Google Scholar 

  19. Shah, D., Gupta, R., Fayazi, A., Hakkani-Tur, D.: Robust zero-shot cross-domain slot filling with example values. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5484–5490 (2019)

    Google Scholar 

  20. Tan, X., Cai, Y., Xu, J., Leung, H.F., Chen, W., Li, Q.: Improving aspect-based sentiment analysis via aligning aspect embedding. Neurocomputing 383, 336–347 (2020)

    Article  Google Scholar 

  21. Tang, M., Zhang, P., He, Y., Xu, Y., Chao, C., Xu, H.: DoSEA: a domain-specific entity-aware framework for cross-domain named entity recogition. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 2147–2156 (2022)

    Google Scholar 

  22. Wang, L., et al.: Bridge to target domain by prototypical contrastive learning and label confusion: re-explore zero-shot learning for slot filling. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 9474–9480 (2021)

    Google Scholar 

  23. Wang, J., Kulkarni, M., Preoţiuc-Pietro, D.: Multi-domain named entity recognition with genre-aware and agnostic inference. In: ACL, pp. 8476–8488 (2020)

    Google Scholar 

  24. Xu, J., et al.: Incorporating context-relevant concepts into convolutional neural networks for short text classification. Neurocomputing 386, 42–53 (2020)

    Article  Google Scholar 

  25. Xu, S., Zhang, X., Wu, Y., Wei, F.: Sequence level contrastive learning for text summarization. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 11556–11565 (2022)

    Google Scholar 

  26. Zhang, X., Yu, B., Wang, Y., Liu, T., Su, T., Xu, H.: Exploring modular task decomposition in cross-domain named entity recognition. In: SIGIR, pp. 301–311 (2022)

    Google Scholar 

  27. Zhang, Y., Zhang, R., Mensah, S., Liu, X., Mao, Y.: Unsupervised sentence representation via contrastive learning with mixing negatives. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 11730–11738 (2022)

    Google Scholar 

  28. Zheng, C., Cai, Y., Xu, J., Leung, H., Xu, G.: A boundary-aware neural model for nested named entity recognition. In: Association for Computational Linguistics (EMNLP-IJCNLP) (2019)

    Google Scholar 

  29. Zheng, J., Chen, H., Ma, Q.: Cross-domain named entity recognition via graph matching. In: Findings of the Association for Computational Linguistics (ACL 2022), pp. 2670–2680 (2022)

    Google Scholar 

Download references

Acknowledgement

This work was supported in part by the National Natural Science Foundation of China under Grant 62076100, in part by the Fundamental Research Funds for the Central Universities, SCUT under Grant x2rjD2220050, in part by the Science and Technology Planning Project of Guangdong Province under Grant 2020B0101100002, in part by the Hong Kong Research Council under Grants PolyU 11204919 and C1031-18G, and in part by the Internal Research from the Hong Kong Polytechnic University under Grant 1.9B0V.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yi Cai .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xu, J., Cai, Y. (2023). Improving Cross-Domain Named Entity Recognition from the Perspective of Representation. In: Wang, X., et al. Database Systems for Advanced Applications. DASFAA 2023. Lecture Notes in Computer Science, vol 13946. Springer, Cham. https://doi.org/10.1007/978-3-031-30678-5_65

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-30678-5_65

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-30677-8

  • Online ISBN: 978-3-031-30678-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics