Abstract
Relation extraction (RE) is an important task in information extraction that has drawn much attention. Although many RE models have achieved impressive performance, their performance drops dramatically when adapting to the new domain and under few-shot scenarios. One reason is that the huge gap in semantic space between different domains makes the model obtain suboptimal representations in the new domain. The other is the inability to learn class-sensitive information with only a few samples, which makes the instances with confusing factors hard to be distinguished. To address these issues, we propose a Contrastive learning-based Fine-Tuning approach with Knowledge Enhancement (CFTKE) for the Domain Adaptation Few-Shot RE task (DAFSRE). Specifically, we fine-tune the model in a contrastive-learning way to refine the semantic space in the new domain, which can bridge the gap between different domains and obtain better representations. To enhance the stability and learning ability of contrastive learning-based fine-tuning, we design the data augmentation mechanism and type-aware networks to enrich the instances and stand out the class-sensitive features. Extensive experiments on the DAFSRE benchmark dataset demonstrate that our approach significantly outperforms the state-of-the-art models (by 2.73% on average).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The pre-trained embeddings of entities and concepts are provided by Zhang et al. [19]
- 2.
U(a, b) is a matrix with the same shape with \(\tilde{\textbf{h}}\) rather than a scalar.
- 3.
References
Cong, X., Yu, B., Liu, T., Cui, S., Tang, H., Wang, B.: Inductive unsupervised domain adaptation for few-shot classification via clustering. In: Hutter, F., Kersting, K., Lijffijt, J., Valera, I. (eds.) ECML PKDD 2020. LNCS (LNAI), vol. 12458, pp. 624–639. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67661-2_37
Dou, C., et al.: Function-words enhanced attention networks for few-shot inverse relation classification. IJCAI (2022)
Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: ICML, pp. 1126–1135. PMLR (2017)
Gao, T., et al.: FewRel 2.0: towards more challenging few-shot relation classification. In: EMNLP, pp. 6250–6255 (2019)
Gao, T., Han, X., Liu, Z., Sun, M.: Hybrid attention-based prototypical networks for noisy few-shot relation classification. In: AAAI, vol. 33, pp. 6407–6414 (2019)
Han, J., Cheng, B., Lu, W.: Exploring task difficulty for few-shot relation extraction. In: EMNLP, pp. 2605–2616 (2021)
Kenton, J.D.M.W.C., Toutanova, L.K.: Bert: pre-training of deep bidirectional transformers for language understanding. In: NAACL, pp. 4171–4186 (2019)
Khosla, P., et al.: Supervised contrastive learning. In: NeurIPS, pp. 18661–18673 (2020)
Liu, F., et al.: From learning-to-match to learning-to-discriminate: global prototype learning for few-shot relation classification. In: Li, S., et al. (eds.) CCL 2021. LNCS (LNAI), vol. 12869, pp. 193–208. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-84186-7_13
Peng, H., et al.: Learning from context or names? An empirical study on neural relation extraction. In: EMNLP, pp. 3661–3672 (2020)
Qu, M., Gao, T., Xhonneux, L.P., Tang, J.: Few-shot relation extraction via Bayesian meta-learning on relation graphs. In: ICML, pp. 7867–7876. PMLR (2020)
Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Soares, L.B., Fitzgerald, N., Ling, J., Kwiatkowski, T.: Matching the blanks: distributional similarity for relation learning. In: ACL, pp. 2895–2905 (2019)
Srivastava, N., et al.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al.: Matching networks for one shot learning. In: Advances in Neural Information Processing Systems, vol. 29 (2016)
Wang, Y., et al.: Learning to decouple relations: few-shot relation classification with entity-guided attention and confusion-aware training. In: COLING, pp. 5799–5809 (2020)
Yang, K., et al.: Enhance prototypical network with text descriptions for few-shot relation classification. In: CIKM, pp. 2273–2276 (2020)
Yang, S., Zhang, Y., Niu, G., Zhao, Q., Pu, S.: Entity concept-enhanced few-shot relation extraction. In: IJCNLP, pp. 987–991 (2021)
Zhang, J., et al.: Knowledge-enhanced domain adaptation in few-shot relation classification. In: SIGKDD, pp. 2183–2191 (2021)
Acknowledgement
This work was supported by No.XDC02050200.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, Y. et al. (2023). Powering Fine-Tuning: Learning Compatible and Class-Sensitive Representations for Domain Adaption Few-shot Relation Extraction. In: Wang, X., et al. Database Systems for Advanced Applications. DASFAA 2023. Lecture Notes in Computer Science, vol 13946. Springer, Cham. https://doi.org/10.1007/978-3-031-30678-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-30678-5_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-30677-8
Online ISBN: 978-3-031-30678-5
eBook Packages: Computer ScienceComputer Science (R0)