Skip to main content
SpringerLink
Log in

Prototypical attention network for few-shot relation classification with entity-aware embedding module

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Relation classification (RC) identifies the semantic relation between entity pairs and plays a critical role in knowledge graph construction and knowledge graph completion. However, insufficient labeled instances of long-tail relations make the training of supervised and distant supervised (DS) relation classification models difficult. Few-shot RC is an effective solution to this problem. At present, metric-based few-shot RC models focus on the representation of relation prototypes and the interaction between instances, ignoring meaningful entity representation and the association of entities and other words in the instance. We propose a prototypical attention network with an entity-aware embedding module (PAN-EAEM) to solve this problem. Firstly, the entity-aware embedding module (EAEM) draws more attention to entity-related words to capture key features. This plug-and-play module can improve the performance of other metric-based models as well. Secondly, the prototypical attention network (PAN) decreases the influence of noise on relation prototype representation by reducing intra-class differences and inter-class ambiguities. Extensive experiments prove that our proposed model obtains state-of-the-art performance on the FewRel dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. http://www.zhuhao.me/fewrel/

  2. https://github.com/ChaoLiu-TJU/PAN-EAEM

References

  1. Agichtein E, Gravano L (2000) Snowball: extracting relations from large plain-text collections. In: Proceedings of the 5th ACM conference on digital libraries. ACM, New York, pp 85– 94

  2. Bi S, Wang Y, Li X, Dong M, Zhu J (2021) Critical direction projection networks for few-shot learning. Appl Intell 51(8):1–14

    Google Scholar 

  3. Chen Q, Zhu X, Ling Z, Wei S, Jiang H, Inkpen D (2017) Enhanced LSTM for natural language inference. In: Proceedings of the 55th annual meeting of the association for computational linguistics. ACL, Stroudsburg, pp 1657–1668

  4. Elsken T, Staffler B, Metzen J H, Hutter F (2020) Meta-learning of neural architectures for few-shot learning. In: Proceedings of the 33rd IEEE conference on computer vision and pattern recognition. IEEE, Piscataway, pp 12362–12372

  5. Fan M, Bai Y, Sun M, Li P (2019) Large margin prototypical network for few-shot relation classification with fine-grained features. In: Proceedings of the 28th ACM international conference on information and knowledge management. ACM, New York, pp 2353–2356

  6. Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the 34th international conference on machine learning. IEEE, Piscataway, pp 1126–1135

  7. Gao T, Han X, Liu Z, Sun M (2019) Hybrid attention-based prototypical networks for noisy few-shot relation classification. In: Proceedings of the 33rd conference on artificial intelligence. AAAI, Menlo Park, pp 6407–6414

  8. Gao T, Han X, Xie R, Liu Z, Lin F, Lin L, Sun M (2020) Neural snowball for few-shot relation learning. In: Proceedings of the 34th AAAI conference on artificial intelligence. AAAI, Menlo Park, pp 7772–7779

  9. Guo Y, Cheung N (2020) Attentive weights generation for few shot learning via information maximization. In: Proceedings of the 33rd IEEE conference on computer vision and pattern recognition. IEEE, Piscataway, pp 13496–13505

  10. Guo Z, Zhang Y, Lu W (2019) Attention guided graph convolutional networks for relation extraction. In: Proceedings of the 57th conference of the association for computational linguistics. ACL, Stroudsburg, pp 241–251

  11. Han X, Zhu H, Yu P, Wang Z, Yao Y, Liu Z, Sun M (2018) Fewrel: a large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. In: Proceedings of the 2018 conference on empirical methods in natural language processing. ACL, Stroudsburg, pp 4803–4809

  12. Jamal MA, Qi G (2019) Task agnostic meta-learning for few-shot learning. In: Proceedings of the 32nd IEEE conference on computer vision and pattern recognition. IEEE, Piscataway, pp 11719–11727

  13. Koch G, Zemel R, Salakhutdinov R (2015) Siamese neural networks for one-shot image recognition. In: Proceedings of the 32nd international conference on machine learning deep learning workshop. ACM, New York

  14. Li W, Wang L, Xu J, Huo J, Gao Y, Luo J (2019a) Revisiting local descriptor based image-to-class measure for few-shot learning. In: Proceedings of the 32nd IEEE conference on computer vision and pattern recognition. IEEE, Piscataway, pp 7260–7268

  15. Li W, Xu J, Huo J, Wang L, Gao Y, Luo J (2019b) Distribution consistency based covariance metric networks for few-shot learning. In: Proceedings of the 33rd AAAI conference on artificial intelligence. AAAI, Menlo Park, pp 8642–8649

  16. Li W, Wang Q, Wu J, Yu Z (2021) Piecewise convolutional neural networks with position attention and similar bag attention for distant supervision relation extraction. Appl Intell 51(6):1–11

    Google Scholar 

  17. Mintz M, Bills S, Snow R, Jurafsky D (2009) Distant supervision for relation extraction without labeled data. In: Proceedings of the 47th annual meeting of the association for computational linguistics and the 4th international joint conference on natural language processing of the AFNLP. ACL, Stroudsburg, pp 1003–1011

  18. Mishra N, Rohaninejad M, Chen X, Abbeel P (2018) A simple neural attentive meta-learner. In: Proceedings of the 6th international conference on learning representations. ICLR, Vancouver

  19. Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing. ACL, Stroudsburg, pp 1532–1543

  20. Qu M, Gao T, Xhonneux LAC, Tang J (2020) Few-shot relation extraction via bayesian meta-learning on relation graphs. In: Proceedings of the 37th international conference on machine learning. ACM, New York, pp 7867–7876

  21. Ren M, Liao R, Fetaya E, Zemel RS (2019) Incremental few-shot learning with attention attractor networks. In: Proceedings of the 32nd international conference on neural information processing systems. MIT Press, Cambridge, pp 5276–5286

  22. Riedel S, Yao L, Mccallum A (2010) Modeling relations and their mentions without labeled text. In: Machine learning and knowledge discovery in databases. Springer, Berlin, pp 148–163

  23. Rusu AA, Rao D, Sygnowski J, Vinyals O, Pascanu R, Osindero S, Hadsell R (2019) Meta-learning with latent embedding optimization. In: Proceedings of the 7th international conference on learning representations. ICLR, Vancouver

  24. Satorras VG, Estrach JB (2018) Few-shot learning with graph neural networks. In: Proceedings of the 6th international conference on learning representations. ICLRer, Vancouver

  25. Schwartz E, Karlinsky L, Shtok J, Harary S, Marder M, Kumar A, Feris RS, Giryes R, Bronstein AM (2018) delta-encoder: an effective sample synthesis method for few-shot object recognition. In: Proceedings of the 32nd conference on neural information processing systems. MIT Press, Cambridge, pp 2850–2860

  26. Shang Y, Huang HY, Mao X, Sun X, Wei W (2020) Are noisy sentences useless for distant supervised relation extraction?. In: Proceedings of the 34th AAAI conference on artificial intelligence. AAAI, Menlo Park, pp 8799–8806

  27. Snell J, Swersky K, Zemel RS (2017) Prototypical networks for few-shot learning. In: Proceedings of the 30th international conference on neural information processing systems. MIT Press, Cambridge, pp 4077–4087

  28. Sun K, Zhang R, Mao Y, Mensah S, Liu X (2020) Relation extraction with convolutional network over learnable syntax-transport graph. In: Proceedings of the 34th AAAI Conference on artificial intelligence. AAAI, Menlo Park, pp 89280–8935

  29. Sun Q, Liu Y, Chua T, Schiele B (2019a) Meta-transfer learning for few-shot learning. In: Proceedings of the 32nd IEEE conference on computer vision and pattern recognition. IEEE, Piscataway, pp 403–412

  30. Sun S, Sun Q, Zhou K, Lv T (2019b) Hierarchical attention prototypical networks for few-shot text classification. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing. ACL, Stroudsburg, pp 476–485

  31. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Proceedings of the 30th international conference on neural information processing systems. MIT Press, Cambridge, pp 5998–6008

  32. Vinyals O, Blundell C, Lillicrap T, Kavukcuoglu K, Wierstra D (2016) Matching networks for one shot learning. In: Proceedings of the 29th international conference on neural information processing systems. MIT Press, Cambridge, pp 3637–3645

  33. Wang X, Girshick RB, Gupta A, He K (2018) Non-local neural networks. In: Proceedings of the 31st IEEE conference on computer vision and pattern recognition. IEEE, Piscataway, pp 7794–7803

  34. Wu S, Fan K, Zhang Q (2019) Improving distantly supervised relation extraction with neural noise converter and conditional optimal selector. In: Proceedings of the 33rd AAAI conference on artificial intelligence. AAAI, Menlo Park, pp 7273–7280

  35. Ye Z, Ling Z (2019) Multi-level matching and aggregation network for few-shot relation classification. In: Proceedings of the 57th conference of the association for computational linguistics. ACL, Stroudsburg, pp 2872–2881

  36. Zeng D, Liu K, Lai S, Zhou G, Zhao J (2014) Relation classification via convolutional deep neural network. In: Proceedings of the 25th international conference on computational linguistics. ACL, Stroudsburg, pp 2335–2344

  37. Zeng D, Liu K, Chen Y, Zhao J (2015) Distant supervision for relation extraction via piecewise convolutional neural networks. In: Proceedings of the 2015 conference on empirical methods in natural language processing. ACL, Stroudsburg, pp 1753–1762

  38. Zhu H, Lin Y, Liu Z, Fu J, Chua T, Sun M (2019) Graph neural networks with generated parameters for relation extraction. In: Proceedings of the 57th conference of the association for computational linguistics. ACL, Stroudsburg, pp 1331–1339

Download references

Acknowledgements

This work is jointly supported by National Natural Science Foundation of China (61877043) and National Natural Science of China (61877044).

Author information

Authors and Affiliations

  1. College of Intelligence and Computing, Tianjin University, Tianjin, China

    Xuewei Li, Chao Liu, Jian Yu, Tianyi Xu, Mankun Zhao, Mei Yu & Ruiguo Yu

  2. Tianjin Key Laboratory of Advanced Networking (TANKLab), Tianjin, China

    Xuewei Li, Chao Liu, Jian Yu, Tianyi Xu, Mankun Zhao, Mei Yu & Ruiguo Yu

  3. Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin, China

    Xuewei Li, Chao Liu, Jian Yu, Tianyi Xu, Mankun Zhao, Mei Yu & Ruiguo Yu

  4. Foreign Language, Literature and Culture Studies Center, Tianjin Foreign Studies University, Tianjin, China

    Hongwei Liu

Authors
  1. Xuewei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jian Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tianyi Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mankun Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hongwei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mei Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Ruiguo Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mei Yu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, X., Liu, C., Yu, J. et al. Prototypical attention network for few-shot relation classification with entity-aware embedding module. Appl Intell 53, 10978–10994 (2023). https://doi.org/10.1007/s10489-022-03677-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03677-z

Keywords

Search

Navigation