skip to main content
10.1145/3539618.3591662acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
research-article

Decoupled Hyperbolic Graph Attention Network for Cross-domain Named Entity Recognition

Published: 18 July 2023 Publication History

Abstract

To address the scarcity of massive labeled data, cross-domain named entity recognition (cross-domain NER) attracts increasing attention. Recent studies focus on decomposing NER into two separate tasks (i.e., entity span detection and entity type classification) to reduce the complexity of the cross-domain transfer. Despite the promising results, there still exists room for improvement. In particular, the rich domain-shared syntactic and semantic information, which are respectively important for entity span detection and entity type classification, are still underutilized. In light of these two challenges, we propose applying graph attention networks (GATs) to encode the above two kinds of information. Moreover, considering that GATs mainly operate in the Euclidean space, which may fail to capture the latent hierarchical relations among words for learning high-quality word representations, we further propose to embed words into Hyperbolic spaces. Finally, a decouple hyperbolic graph attention network (DH-GAT) is introduced for cross-domain NER. Empirical results on 10 domain pairs show that DH-GAT achieves state-of-the-art performance on several standard metrics, and further analyses are presented to better understand each component's effectiveness.

References

[1]
Ines Chami, Zhitao Ying, Christopher Ré, and Jure Leskovec. 2019. Hyperbolic Graph Convolutional Neural Networks. Proc. of NeurIPS (2019).
[2]
Wei Chen, Songqiao Han, and Hailiang Huang. 2022. An Empirical Cross Domain-Specific Entity Recognition with Domain Vector. In Proc. of CIKM.
[3]
Leyang Cui, Yu Wu, Jian Liu, Sen Yang, and Yue Zhang. 2021. Template-Based Named Entity Recognition Using BART. In Proc. of ACL Findings.
[4]
Timothy Dozat and Christopher D. Manning. 2018. Simpler but More Accurate Semantic Dependency Parsing. In Proc. of ACL.
[5]
Besnik Fetahu, Anjie Fang, Oleg Rokhlenko, and Shervin Malmasi. 2021. Gazetteer enhanced named entity recognition for code-mixed web queries. In Proc. of SIGIR.
[6]
Tsu-Jui Fu, Peng-Hsuan Li, and Wei-Yun Ma. 2019. Graphrel: Modeling text as relational graphs for joint entity and relation extraction. In Proc. of ACL.
[7]
Octavian-Eugen Ganea, Gary Bécigneul, and Thomas Hofmann. 2018. Hyperbolic neural networks. In Proc. of ICONIP.
[8]
Caglar Gulcehre, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia, Victor Bapst, David Raposo, Adam Santoro, et al. 2018. Hyperbolic Attention Networks. In Proc. of ICLR.
[9]
Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. Proc. of NeurIPS (2017).
[10]
Sigurdur Helgason. 1979. Differential geometry, Lie groups, and symmetric spaces. Academic press.
[11]
Jinpeng Hu, He Zhao, Dan Guo, Xiang Wan, and Tsung-Hui Chang. 2022. A Label-Aware Autoregressive Framework for Cross-Domain NER. In Proc. of ACL Findings.
[12]
Jiaxin Huang, Yu Meng, and Jiawei Han. 2022. Few-Shot Fine-Grained Entity Typing with Automatic Label Interpretation and Instance Generation. In Proc. of KDD.
[13]
Chen Jia, Xiaobo Liang, and Yue Zhang. 2019. Cross-domain NER using cross-domain language modeling. In Proc. of ACL.
[14]
Chen Jia and Yue Zhang. 2020. Multi-cell compositional LSTM for NER domain adaptation. In Proc. of ACL.
[15]
Meihuizi Jia, Xin Shen, Lei Shen, Jinhui Pang, Lejian Liao, Yang Song, Meng Chen, and Xiaodong He. 2022. Query Prior Matters: A MRC Framework for Multimodal Named Entity Recognition. In Proc. of ACM MM.
[16]
Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proc. of NAACL.
[17]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[18]
Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In Proc. of ICLR.
[19]
Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural Architectures for Named Entity Recognition. In Proc. of NAACL.
[20]
Changki Lee, Yi-Gyu Hwang, and Myung-Gil Jang. 2007. Fine-grained named entity recognition and relation extraction for question answering. In Proc. of SIGIR.
[21]
Jeffrey M Lee, Bennett Chow, Sun-Chin Chu, David Glickenstein, Christine Guenther, James Isenberg, Tom Ivey, Dan Knopf, Peng Lu, Feng Luo, et al. 2009. Manifolds and differential geometry. Topology (2009).
[22]
Ji Young Lee, Franck Dernoncourt, and Peter Szolovits. 2018. Transfer Learning for Named-Entity Recognition with Neural Networks. In Proc. of LREC.
[23]
Jingye Li and et al. 2022. Unified named entity recognition as word-word relation classification. In Proc. of AAAI.
[24]
Bill Yuchen Lin and Wei Lu. 2018. Neural Adaptation Layers for Cross-domain Named Entity Recognition. In Proc. of EMNLP.
[25]
Qi Liu, Maximilian Nickel, and Douwe Kiela. 2019. Hyperbolic graph neural networks. Proc. of NeurIPS (2019).
[26]
Zihan Liu, Yan Xu, Tiezheng Yu, Wenliang Dai, Ziwei Ji, Samuel Cahyawijaya, Andrea Madotto, and Pascale Fung. 2021. Crossner: Evaluating cross-domain named entity recognition. In Proc. of AAAI.
[27]
Di Lu, Leonardo Neves, Vitor Carvalho, Ning Zhang, and Heng Ji. 2018. Visual attention model for name tagging in multimodal social media. In Proc. of ACL.
[28]
Ruotian Ma and et al. 2022. Searching for Optimal Subword Tokenization in Cross-domain NER. In Proc. of IJCAI.
[29]
Christopher D Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proc. of ACL.
[30]
Hoang-Van Nguyen, Francesco Gelli, and Soujanya Poria. 2021. DOZEN: cross-domain zero shot named entity recognition with knowledge graph. In Proc. of SIGIR.
[31]
Maximillian Nickel and Douwe Kiela. 2017. Poincaré embeddings for learning hierarchical representations. Proc. of NeurIPS (2017).
[32]
Maximillian Nickel and Douwe Kiela. 2018. Learning continuous hierarchies in the lorentz model of hyperbolic geometry. In Proc. of ICML.
[33]
Qi Peng, Changmeng Zheng, Yi Cai, Tao Wang, Haoran Xie, and Qing Li. 2021. Unsupervised cross-domain named entity recognition using entity-aware adversarial training. Neural Networks (2021).
[34]
Juan Diego Rodriguez, Adam Caldwell, and Alex Liu. 2018. Transfer learning for entity recognition of novel classes. In Proc. of COLING.
[35]
Erik Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition. In Proc. of AACL.
[36]
Yongliang Shen, Xiaobin Wang, Zeqi Tan, Guangwei Xu, Pengjun Xie, Fei Huang, Weiming Lu, and Yueting Zhuang. 2022. Parallel Instance Query Network for Named Entity Recognition. In Proc. of ACL.
[37]
Minghao Tang, Peng Zhang, Yongquan He, Yongxiu Xu, Chengpeng Chao, and Hongbo Xu. 2022. DoSEA: A Domain-specific Entity-aware Framework for Cross-Domain Named Entity Recogition. In Proc. of COLING.
[38]
Meihan Tong, Shuai Wang, Bin Xu, Yixin Cao, Minghui Liu, Lei Hou, and Juanzi Li. 2021. Learning from Miscellaneous Other-Class Words for Few-shot Named Entity Recognition. In Proc. of ACL.
[39]
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In Proc. of ICLR.
[40]
Jing Wang, Mayank Kulkarni, and Daniel Preoţiuc-Pietro. 2020. Multi-domain named entity recognition with genre-aware and agnostic inference. In Proc. of ACL.
[41]
Linyi Yang, Lifan Yuan, Leyang Cui, Wenyang Gao, and Yue Zhang. 2022. FactMix: Using a Few Labeled In-domain Examples to Generalize to Cross-domain Named Entity Recognition. In Proc. of COLING.
[42]
Yaosheng Yang, Wenliang Chen, Zhenghua Li, Zhengqiu He, and Min Zhang. 2018. Distantly supervised NER with partial annotation learning and reinforcement learning. In Proc. of COLING.
[43]
Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, Will Hamilton, and Jure Leskovec. 2018. Hierarchical graph representation learning with differentiable pooling. Proc. of NeurIPS (2018).
[44]
Li Yuan, Yi Cai, Jin Wang, and Qing Li. 2023. Joint Multimodal Entity-Relation Extraction Based on Edge-enhanced Graph Alignment Network and Word-pair Relation Tagging. Proc. of AAAI (2023).
[45]
Tao Zhang, Congying Xia, S Yu Philip, Zhiwei Liu, and Shu Zhao. 2021. PDALN: Progressive domain adaptation over a pre-trained model for low-resource cross-domain named entity recognition. In Proc. of EMNLP.
[46]
Xinghua Zhang, Bowen Yu, Yubin Wang, Tingwen Liu, Taoyu Su, and Hongbo Xu. 2022. Exploring Modular Task Decomposition in Cross-domain Named Entity Recognition. In Proc. of SIGIR.
[47]
Yiding Zhang, Xiao Wang, Chuan Shi, Xunqiang Jiang, and Yanfang Ye. 2021. Hyperbolic graph attention network. IEEE Transactions on Big Data (2021).
[48]
Yu Zhang, Houquan Zhou, and Zhenghua Li. 2021. Fast and accurate neural CRF constituency parsing. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence.
[49]
Junhao Zheng, Haibin Chen, and Qianli Ma. 2022. Cross-domain Named Entity Recognition via Graph Matching. In Proc. of ACL Findings.
[50]
Enwei Zhu and Jinpeng Li. 2022. Boundary Smoothing for Named Entity Recognition. In Proc. of ACL.

Cited By

View all
  • (2024)Exogenous and Endogenous Data Augmentation for Low-Resource Complex Named Entity RecognitionProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657754(630-640)Online publication date: 10-Jul-2024
  • (2024)Prioritizing Potential Wetland Areas via Region-to-Region Knowledge Transfer and Adaptive Propagation2024 IEEE International Conference on Big Data (BigData)10.1109/BigData62323.2024.10825821(1956-1963)Online publication date: 15-Dec-2024
  • (2024)Improving the Text Convolution Mechanism with Large Language Model for Review-Based Recommendation2024 IEEE International Conference on Big Data (BigData)10.1109/BigData62323.2024.10825014(6977-6981)Online publication date: 15-Dec-2024

Index Terms

  1. Decoupled Hyperbolic Graph Attention Network for Cross-domain Named Entity Recognition

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval
    July 2023
    3567 pages
    ISBN:9781450394086
    DOI:10.1145/3539618
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 18 July 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. domain adaptation
    2. hyperbolic embedding
    3. named entity recognition

    Qualifiers

    • Research-article

    Funding Sources

    • CAAI-Huawei MindSpore Open Fund
    • the Science and Technology Planning Project of Guangdong Province
    • the National Natural Science Foundation of China
    • Fundamental Research Funds for the Central Universities, SCUT

    Conference

    SIGIR '23
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 792 of 3,983 submissions, 20%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)134
    • Downloads (Last 6 weeks)13
    Reflects downloads up to 30 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Exogenous and Endogenous Data Augmentation for Low-Resource Complex Named Entity RecognitionProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657754(630-640)Online publication date: 10-Jul-2024
    • (2024)Prioritizing Potential Wetland Areas via Region-to-Region Knowledge Transfer and Adaptive Propagation2024 IEEE International Conference on Big Data (BigData)10.1109/BigData62323.2024.10825821(1956-1963)Online publication date: 15-Dec-2024
    • (2024)Improving the Text Convolution Mechanism with Large Language Model for Review-Based Recommendation2024 IEEE International Conference on Big Data (BigData)10.1109/BigData62323.2024.10825014(6977-6981)Online publication date: 15-Dec-2024

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media