Skip to main content
SpringerLink
Log in

DPNet: domain-aware prototypical network for interdisciplinary few-shot relation classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

As few-shot relation classification requires less data to train the neural network models, high costs related to data collection and labeling are eliminated. Compared with the traditional few-shot task, the interdisciplinary few-shot relation classification faces many semantic gap issues (e.g., common sense, subject terminology and insufficient migratable information), bringing the linguistic bias between the testing phase and the training phase. Previous methods mainly focus on the training phase itself and neglect the inherent syntactic and semantic heterogeneity in terms of gap issues. In this paper, we propose a Domain-Aware Prototypical Network (DPNet) to cast the interdisciplinary few-shot relation classification as a covariate shift task. First, the relational keyword masking mechanism, which masks the keywords with a certain probability, is adopted to learn the contextual structure information. Second, to reduce the influence of subject-specific terminology, the adaptive word mover’s distance (WMD) is proposed to calculate the similarity among test samples by weighting each word in the sentence. Third, the knowledge distillation strategy integrates each category distribution characteristic into a prototypical network, which supplements additional information for metric learning in task-adaptive feature sub-space. We conduct various experiments on two widely used benchmarks (FewRel 2.0 and DDI-13 dataset). Experimental results show that our model outperforms related methods by a significant margin.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Han X, Zhu H, Yu P, Wang Z, Yao Y, Liu Z, Sun M (2018) FewRel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. In: Proceedings of the 2018 conference on empirical methods in natural language processing. Association for Computational Linguistics, Brussels, Belgium, pp 4803–4809

  2. Wu F, Smith JS, Lu W, Pang C, Zhang B (2020) Attentive prototype few-shot learning with capsule network-based embedding. Proceedings of the European Conference on Computer Vision 12373:237–253

    Google Scholar 

  3. Vinyals O, Blundell C, Lillicrap T, kavukcuoglu, Wierstra D (2016) Matching networks for one shot learning. In: Lee D, Sugiyama M, Luxburg U, Guyon I, Garnett R (eds) proceedings of the neural information processing systems conference, pp 3630–3638

  4. Cong X, Yu B, Liu T, Cui S, Tang H, Wang B (2020) Inductive unsupervised domain adaptation for few-shot classification via clustering. In: Proceedings of the 2020 European conference on machine learning and principles and practice of knowledge discovery in database

  5. Gao T, Han X, Liu Z, Sun M (2019) Hybrid attention-based prototypical networks for noisy few-shot relation classification. Proceedings of the AAAI Conference on Artificial Intelligence 33(01):6407–6414

    Article  Google Scholar 

  6. Devlin J, Chang M-W, Lee K, Toutanova K (June 2019) BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers). Association for Computational Linguistics, Minneapolis, Minnesota, pp 4171–4186

  7. Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (1907) Roberta: A robustly optimized bert pretraining approach, 2019. arXiv:1907.11692 364

  8. Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. J Mach Learn Res 21(140):1–67

    MathSciNet  MATH  Google Scholar 

  9. Lewis M, Liu Y, Goyal N, Ghazvininejad M, Mohamed A, Levy O, Stoyanov V, Zettlemoyer L (2020) BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In: Proceedings of the 58th annual meeting of the association for computational linguistics. Association for computational linguistics, Online, pp 7871–7880

  10. Snell J, Swersky K, Zemel RS (2017) Prototypical networks for few-shot learning. In: Guyon I, von Luxburg U, Bengio S, Wallach H M, Fergus R, Vishwanathan S V N, Garnett R (eds) Proceedings of the 2017 conference and workshop on neural information processing systems, pp 4077–4087

  11. Wen W, Liu Y, Ouyang C, Lin Q, Chung T (2021) Enhanced prototypical network for few-shot relation extraction. Information Processing & Management 58(4):102596

    Article  Google Scholar 

  12. Huang P-S, Wang C, Singh R, Yih W, He X (2018) Natural language to structured query generation via meta-learning. In: Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 2 (short papers). Association for Computational Linguistics, New Orleans, Louisiana, pp 732–738

  13. Yu C, Han J, Zhang H, Ng W (2020) Hypernymy detection for low-resource languages via meta learning. In: Proceedings of the 58th annual meeting of the association for computational linguistics. Association for Computational Linguistics, pp 3651–3656

  14. Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. Proceedings of the 2017 International Conference on Machine Learning, abs/1703.03400

  15. Wohlwend J, Elenberg ER, Altschul S, Henry S, Lei T (2019) Metric learning for dynamic text classification. In: Proceedings of the 2nd workshop on deep learning approaches for low-resource NLP (DeepLo 2019). Association for Computational Linguistics, Hong Kong, China, pp 143–152

  16. Xiong S, Zhang Y, Ji D, Lou Y (2016) Distance metric learning for aspect phrase grouping. In: Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers. The COLING 2016 Organizing Committee, Osaka, Japan, pp 2492–2502

  17. Bengio Y, Bengio S, Cloutier J (1991) Learning a synaptic learning rule. In: Proceedings of the IJCNN-91-seattle international joint conference on neural networks, vol ii, pp 969 vol.2–

  18. Ravi S, Larochelle H (2017) Optimization as a model for few-shot learning.. In: Proceedings of the 2017 international conference on learning representations

  19. Li Z, Zhou F, Chen F, Li H (2017) Meta-sgd: Learning to learn quickly for few shot learning. Computing Research Repository, abs/1707.09835

  20. Bansal T, Jha R, Munkhdalai T, McCallum A (November 2020) Self-supervised meta-learning for few-shot natural language classification tasks. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Online, pp 522–534

  21. Rusu AA, Rao D, Sygnowski J, Vinyals O, Pascanu R, Osindero S, Hadsell R (2018) Meta-learning with latent embedding optimization. CoRR abs/1807.05960

  22. Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: Precup D, Teh YW (eds) Proceedings of the 34th international conference on machine learning. Proceedings of Machine Learning Research, vol 70. PMLR, pp 1126–1135

  23. Lee K, Maji S, Ravichandran A, Soatto S (2019) Meta-learning with differentiable convex optimization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

  24. Hidalgo, Ángel Casado, Moreno P, Valent?n, Luis (2021) Using meta-learning to predict student performance in virtual learning environments. In: Applied intelligence

  25. Dou Z-Y, Yu K, Anastasopoulos A (November 2019) Investigating meta-learning algorithms for low-resource natural language understanding tasks. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China, pp 1192–1197

  26. Satorras VG, Estrach JB (2018) Few-shot learning with graph neural networks. In: Proceedings of the 2018 international conference on learning representations

  27. Mishra N, Rohaninejad M, Chen X, Abbeel P (2018) A simple neural attentive meta-learner.. In: Proceedings of the 2018 international conference on learning representations. OpenReview.net

  28. Geng X, Chen X, Zhu KQ, Shen L, Zhao Y (2020) Mick: A meta-learning framework for few-shot relation classification with small training data. Proceedings of the 29th ACM International Conference on Information & Knowledge Management

  29. Wang Y, Bao J, Liu G, Wu Y, He X, Zhou B, Zhao T (2020) Learning to decouple relations: Few-shot relation classification with entity-guided attention and confusion-aware training. In: Proceedings of the 28th International Conference on Computational Linguistics. International committee on computational linguistics, Barcelona, Spain (Online), pp 5799–5809

  30. Dong B, Yao Y, Xie R, Gao T, Han X, Liu Z, Lin F, Lin L (2019) Meta-information guided meta-learning for few-shot relation classification. In: Proceedings of the 28th International Conference on Computational Linguistics. International committee on computational linguistics, Barcelona, Spain (Online), pp 1594–1605

  31. Qu M, Gao T, Xhonneux L-P, Tang J (2020) Few-shot relation extraction via bayesian meta-learning on relation graphs. In: III HD, Singh A (eds) Proceedings of the 37th international conference on machine learning. Proceedings of Machine Learning Research, vol 119. PMLR, pp 7867–7876

  32. Koch GR (2015) Siamese neural networks for one-shot image recognition

  33. Fan M, Bai Y, Sun M, Li P (2019) Large margin prototypical network for few-shot relation classification with fine-grained features. In: Zhu W, Tao D, Cheng X, Cui P, Rundensteiner E A, Carmel D, He Q, Yu JX (eds) Proceedings of the 2019 ACM international conference on information and knowledge management. ACM, pp 2353–2356

  34. Ren H, Cai Y, Chen X, Wang G, Li Q (2020) A two-phase prototypical network model for incremental few-shot relation classification. In: Proceedings of the 28th international conference on computational linguistics. International Committee on Computational Linguistics, Barcelona, Spain (Online), pp 1618–1629

  35. Ding N, Wang X, Fu Y, Xu G, Wang R, Xie P, Shen Y, Huang F, Zheng H-T, Zhang R (2021) Prototypical representation learning for relation extraction. In: Proceedings of the 2021 international conference on learning representations

  36. Wang, Fei, LiChen, ZengZhen, XuKe, Cheng, LiuYanjun (2021) Cornerstone network with feature extractor: a metric-based few-shot model for chinese natural sign language. In: Applied intelligence

  37. Ye Z-X, Ling Z-H (July 2019) Multi-level matching and aggregation network for few-shot relation classification. In: Proceedings of the 57th annual meeting of the association for computational linguistics. Association for Computational Linguistics, Florence, Italy, pp 2872–2881

  38. Xie Y, Xu H, Li J, Yang C, Gao K (2020) Heterogeneous graph neural networks for noisy few-shot relation classification. Knowl Based Syst 194:105548

    Article  Google Scholar 

  39. Wu L, Zhang H-P, Yang Y, Liu X, Gao K (2020) Dynamic prototype selection by fusing attention mechanism for few-shot relation classification. In: Nguyen N T, Jearanaitanakij K, Selamat A, Trawinski B, Chittayasothorn S (eds) ACIIDS (1). Lecture Notes in Computer Science. Springer, pp 431–441

  40. Sung F, Yang Y, Zhang L, Xiang T, Torr PHS, Hospedales TM (2018) Learning to compare: Relation network for few-shot learning. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 1199–1208

  41. Wu Z, Li Y, Guo L, Jia K (2019) Parn: Position-aware relation networks for few-shot learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

  42. Wang Z, Zhao Y, Li J, Tian Y (2020) Cooperative bi-path metric for few-shot learning. Proceedings of the 28th ACM International Conference on Multimedia

  43. Hinton G, Vinyals O, Dean J (2014) Distilling the knowledge in a neural network. Proceedings of the 2014 Conference and Workshop on Neural Information Processing Systems

  44. Kusner MJ, Sun Y, Kolkin NI, Weinberger KQ (2015) From word embeddings to document distances. In: Proceedings of the 32Nd international conference on international conference on machine learning - volume 37. ICML’15. JMLR.org, pp 957– 966

  45. Baldini Soares L, FitzGerald N, Ling J, Kwiatkowski T (July 2019) Matching the blanks: Distributional similarity for relation learning. In: Proceedings of the 57th annual meeting of the association for computational linguistics. Association for Computational Linguistics, Florence, Italy, pp 2895–2905

  46. Peng H, Gao T, Han X, Lin Y, Li P, Liu Z, Sun M, Zhou J (2020) Learning from Context or Names? An Empirical Study on Neural Relation Extraction. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Online, pp 3661–3672

Download references

Author information

Authors and Affiliations

  1. Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, 100190, China

    Bo Lv, Li Jin, Xiaoyu Li, Xian Sun, Zhi Guo, Zequn Zhang & Shuchao Li

  2. Key Laboratory of Network Information System Technology, Institute of Electronics, Chinese Academy of Sciences, Beijing, 100190, China

    Bo Lv, Li Jin, Xiaoyu Li, Xian Sun, Zhi Guo, Zequn Zhang & Shuchao Li

  3. School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing, 100190, China

    Bo Lv

Authors
  1. Bo Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Li Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoyu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xian Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhi Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zequn Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Shuchao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Jin.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lv, B., Jin, L., Li, X. et al. DPNet: domain-aware prototypical network for interdisciplinary few-shot relation classification. Appl Intell 52, 15718–15733 (2022). https://doi.org/10.1007/s10489-022-03210-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03210-2

Keywords

Search

Navigation