Abstract
Few-shot relation classification has attracted great attention recently, and is regarded as an effective way to tackle the long-tail problem in relation classification. Most previous works on few-shot relation classification are based on learning-to-match paradigms, which focus on learning an effective universal matcher between the query and one target class prototype based on inner-class support sets. However, the learning-to-match paradigm focuses on capturing the similarity knowledge between query and class prototype, while fails to consider discriminative information between different candidate classes. Such information is critical especially when target classes are highly confusing and domain shifting exists between training and testing phases. In this paper, we propose the Global Transformed Prototypical Networks (GTPN), which learns to build a few-shot model to directly discriminate between the query and all target classes with both inner-class local information and inter-class global information. Such learning-to-discriminate paradigm can make the model concentrate more on the discriminative knowledge between all candidate classes, and therefore leads to better classification performance. We conducted experiments on standard FewRel benchmarks. Experimental results show that GTPN achieves very competitive performance on few-shot relation classification and reached the best performance on the official leaderboard of FewRel 2.0 (https://thunlp.github.io/2/fewrel2_da.html).
F. Liu, part of the work was done during an internship at Baidu.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Andrychowicz, M., et al: Learning to learn by gradient descent by gradient descent. In: NeurIPS (2016). http://dl.acm.org/citation.cfm?id=3157382.3157543
Baldini Soares, L., FitzGerald, N., Ling, J., Kwiatkowski, T.: Matching the blanks: distributional similarity for relation learning. In: ACL(2019). https://www.aclweb.org/anthology/P19-1279
Bunescu, R., Mooney, R.: A shortest path dependency kernel for relation extraction. In: EMNLP (2005). https://www.aclweb.org/anthology/H05-1091
Cong, X., Yu, B., Liu, T., Cui, S., Tang, H., Wang, B.: Inductive unsupervised domain adaptation for few-shot classification via clustering. In: ECML-PKDD (2020). https://arxiv.org/abs/2006.12816
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL (2019). https://www.aclweb.org/anthology/N19-1423
Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: ICML (2017). http://dl.acm.org/citation.cfm?id=3305381.3305498
Gao, T., Han, X., Liu, Z., Sun, M.: Hybrid attention-based prototypical networks for noisy few-shot relation classification. In: AAAI (2019). https://aaai.org/ojs/index.php/AAAI/article/view/4604/4482
Gao, T., et al.: FewRel 2.0: Towards more challenging few-shot relation classification. In: EMNLP-IJCNLP (2019). https://www.aclweb.org/anthology/D19-1649
Gidaris, S., Bursuc, A., Komodakis, N., Perez, P., Cord, M.: Boosting few-shot visual learning with self-supervision. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019
Han, X., et al.: FewRel: a large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. In: EMNLP (2018). https://www.aclweb.org/anthology/D18-1514
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Koch, G., Zemel, R., Salakhutdinov, R.: Siamese neural networks for one-shot image recognition. In: ICML Deep Learning Workshop, vol. 2 (2015). https://sites.google.com/site/deeplearning2015/37.pdf?attredirects=0
Liu, Y., et al..: Learning to propagate labels: transductive propagation network for few-shot learning. In: ICLR (2019). https://openreview.net/forum?id=SyVuRiC5K7
Mishra, N., Rohaninejad, M., Chen, X., Abbeel, P.: A simple neural attentive meta-learner. In: ICLR (2018). https://openreview.net/forum?id=B1DmUzWAW
Munkhdalai, T., Yu, H.: Meta networks. In: ICML (2017). http://proceedings.mlr.press/v70/munkhdalai17a.html
Nguyen, T.V.T., Moschitti, A., Riccardi, G.: Convolution kernels on constituent, dependency and sequential structures for relation extraction. In: EMNLP (2009). https://www.aclweb.org/anthology/D09-1143
Oreshkin, B., Rodríguez López, P., Lacoste, A.: Tadam: Task dependent adaptive metric for improved few-shot learning. In: NeurIPS (2018). http://papers.nips.cc/paper/7352-tadam-task-dependent-adaptive-metric-for-improved-few-shot-learning.pdf
Peng, H., et al.: Learning from context or names? An empirical study on neural relation extraction. In: EMNLP (2020). https://www.aclweb.org/anthology/2020.emnlp-main.298
Peng, N., Poon, H., Quirk, C., Toutanova, K., Yih, W.t.: Cross-sentence n-ARV relation extraction with graph LSTMs. In: TACL (2017). https://www.aclweb.org/anthology/Q17-1008
Qian, L., Zhou, G., Kong, F., Zhu, Q., Qian, P.: Exploiting constituent dependencies for tree kernel-based semantic relation extraction. In: COLING (2008). https://www.aclweb.org/anthology/C08-1088
Qu, M., Gao, T., Xhonneux, L.P.A.C., Tang, J.: Few-shot relation extraction via Bayesian meta-learning on relation graphs. In: ICML (2020), https://proceedings.icml.cc/paper/2020/file/99607461cdb9c26e2bd5f31b12dcf27a-Paper.pdf
Ravi, S., Larochelle, H.: Optimization as a model for few-shot learning. In: ICLR (2017)
Santoro, A., Bartunov, S., Botvinick, M.M., Wierstra, D., Lillicrap, T.P.: One-shot learning with memory-augmented neural networks. ArXiv abs/1605.06065 (2016)
Satorras, V.G., Estrach, J.B.: Few-shot learning with graph neural networks. In: International Conference on Learning Representations (2018). https://openreview.net/forum?id=BJj6qGbRW
Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: NeurIPS (2017). http://papers.nips.cc/paper/6996-prototypical-networks-for-few-shot-learning.pdf
Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H., Hospedales, T.M.: Learning to compare: Relation network for few-shot learning. In: CVPR (2018)
Tai, K.S., Socher, R., Manning, C.D.: Improved semantic representations from tree-structured long short-term memory networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). pp. 1556–1566. Association for Computational Linguistics, Beijing, China, July 2015. https://doi.org/10.3115/v1/P15-1150, https://www.aclweb.org/anthology/P15-1150
Vaswani, A., et al.: Attention is all you need. In: NeurIPS (2017), http://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. In: ICLR (2018), https://openreview.net/forum?id=rJXMpikCZ
Vinyals, O., Blundell, C., Lillicrap, T., kavukcuoglu, k., Wierstra, D.: Matching networks for one shot learning. In: NeurIPS (2016). http://papers.nips.cc/paper/6385-matching-networks-for-one-shot-learning.pdf
Wang, L., Cao, Z., de Melo, G., Liu, Z.: Relation classification via multi-level attention CNNs. In: ACL (2016). https://www.aclweb.org/anthology/P16-1123
Yang, S., Liu, L., Xu, M.: Free lunch for few-shot learning: Distribution calibration. In: International Conference on Learning Representations (2021). https://openreview.net/forum?id=JWOiYxMG92s
Ye, Z.X., Ling, Z.H.: Multi-level matching and aggregation network for few-shot relation classification. In: ACL (2019). https://www.aclweb.org/anthology/P19-1277
Zeng, D., Liu, K., Lai, S., Zhou, G., Zhao, J.: Relation classification via convolutional deep neural network. In: COLING (2014). https://www.aclweb.org/anthology/C14-1220
Zhou, L., Cui, P., Jia, X., Yang, S., Tian, Q.: Learning to select base classes for few-shot classification. In: CVPR (2020)
Zhou, P., et al. : Attention-based bidirectional long short-term memory networks for relation classification. In: ACL (2016). https://www.aclweb.org/anthology/P16-2034
Acknowledgements
This work is supported by the National Key R&D Program of China under Grant 2018YFB1005100.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, F. et al. (2021). From Learning-to-Match to Learning-to-Discriminate: Global Prototype Learning for Few-shot Relation Classification. In: Li, S., et al. Chinese Computational Linguistics. CCL 2021. Lecture Notes in Computer Science(), vol 12869. Springer, Cham. https://doi.org/10.1007/978-3-030-84186-7_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-84186-7_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-84185-0
Online ISBN: 978-3-030-84186-7
eBook Packages: Computer ScienceComputer Science (R0)