Abstract
Meta-learning has shown remarkable success in few-shot learning, and a popular metric-based meta-learning method known as prototypical network has gained widespread adoption for addressing few-shot text classification tasks. However, its effectiveness is hampered by the reliance on limited labeled samples to define class prototypes, which may not accurately reflect the true class distribution, especially given the sparsity of textual data. This misalignment can consequently reduce the performance of few-shot text classification. To address this problem, we propose an optimization method for the prototypical network named LP-PN by leveraging a semi-supervised learning technique known as label propagation. LP-PN utilizes unlabeled samples from query set to optimize the representation of corresponding class prototypes, thus aligning prototypes more closely with the actual class distribution. Furthermore, to overcome the limitations of static distance metrics that fail to capture class differences, we incorporate a dynamic distance metric based on the attention mechanism in LP-PN. We evaluate our method across four benchmark datasets, and the results show that LP-PN demonstrates competitive performance compared with recent few-shot text classification methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bao, Y., Wu, M., Chang, S., Barzilay, R.: Few-shot text classification with distributional signatures. In: 8th International Conference on Learning Representations, ICLR 2020 (2020)
Chung, F.R.K.: Spectral Graph Theory. American Mathematical Soc., Providence (1997)
Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the 34th International Conference on Machine Learning, ICML 2017, pp. 1126–1135 (2017)
Geng, R., Li, B., Li, Y., Zhu, X., Jian, P., Sun, J.: Induction networks for few-shot text classification. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, pp. 3902–3911 (2019)
Han, C., Fan, Z., Zhang, D., Qiu, M., Gao, M., Zhou, A.: Meta-learning adversarial domain adaptation network for few-shot text classification. In: Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, pp. 1664–1673 (2021)
He, R., McAuley, J.J.: Ups and downs: modeling the visual evolution of fashion trends with one-class collaborative filtering. In: Proceedings of the 25th International Conference on World Wide Web, WWW 2016, pp. 507–517 (2016)
Hong, S.K., Jang, T.Y.: LEA: meta knowledge-driven self-attentive document embedding for few-shot text classification. In: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022, pp. 99–106 (2022)
Hospedales, T.M., Antoniou, A., Micaelli, P., Storkey, A.J.: Meta-learning in neural networks: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 44(9), 5149–5169 (2022)
Iscen, A., Tolias, G., Avrithis, Y., Chum, O.: Label propagation for deep semi-supervised learning. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, pp. 5070–5079 (2019)
Joulin, A., Grave, E., Bojanowski, P., Douze, M., Jégou, H., Mikolov, T.: FastText.zip: compressing text classification models. CoRR abs/1612.03651 (2016)
Kim, H.H., Woo, D., Oh, S.J., Cha, J., Han, Y.: ALP: data augmentation using lexicalized PCFGs for few-shot text classification. In: Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, pp. 10894–10902 (2022)
Lang, K.: NewsWeeder: learning to filter netnews. In: Machine Learning, Proceedings of the Twelfth International Conference on Machine Learning, pp. 331–339 (1995)
Lee, H., Li, S., Vu, T.: Meta learning for natural language processing: a survey. In: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022, pp. 666–684 (2022)
Lei, S., Zhang, X., He, J., Chen, F., Lu, C.: TART: improved few-shot text classification using task-adaptive reference transformation. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, pp. 11014–11026 (2023)
Lewis, D.: Reuters-21578 text categorization collection (1997). https://doi.org/10.24432/C52G6M
Li, B., Li, Y., Zhang, X.: A survey on Laplacian eigenmaps based manifold learning methods. Neurocomputing 335, 336–351 (2019)
van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)
Minaee, S., Kalchbrenner, N., Cambria, E., Nikzad, N., Chenaghlu, M., Gao, J.: Deep learning-based text classification: a comprehensive review. ACM Comput. Surv. 54(3), 62:1–62:40 (2022)
Mishra, N., Rohaninejad, M., Chen, X., Abbeel, P.: A simple neural attentive meta-learner. In: 6th International Conference on Learning Representations, ICLR 2018, Conference Track Proceedings (2018)
Misra, R.: News category dataset. CoRR abs/2209.11429 (2022)
Nichol, A., Achiam, J., Schulman, J.: On first-order meta-learning algorithms. CoRR abs/1803.02999 (2018)
Santoro, A., Bartunov, S., Botvinick, M.M., Wierstra, D., Lillicrap, T.P.: Meta-learning with memory-augmented neural networks. In: Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, vol. 48, pp. 1842–1850 (2016)
Snell, J., Swersky, K., Zemel, R.S.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, pp. 4077–4087 (2017)
Sun, Y., Zheng, Y., Hao, C., Qiu, H.: NSP-BERT: a prompt-based few-shot learner through an original pre-training task - - next sentence prediction. In: Proceedings of the 29th International Conference on Computational Linguistics, COLING 2022, pp. 3233–3250 (2022)
Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H.S., Hospedales, T.M.: Learning to compare: relation network for few-shot learning. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, pp. 1199–1208 (2018)
Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., Wierstra, D.: Matching networks for one shot learning. In: Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, pp. 3630–3638 (2016)
Wei, J.W., Zou, K.: EDA: easy data augmentation techniques for boosting performance on text classification tasks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, pp. 6381–6387 (2019)
Yu, M., et al.: Diverse few-shot text classification with multiple metrics. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, Volume 1 (Long Papers), pp. 1206–1215 (2018)
Zhang, H., Zhang, X., Huang, H., Yu, L.: Prompt-based meta-learning for few-shot text classification. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, pp. 1342–1357 (2022)
Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. In: Advances in Neural Information Processing Systems 16 [Neural Information Processing Systems, NIPS 2003], pp. 321–328 (2003)
Acknowledgments
This work is supported by the National Natural Science Foundation of China (No. 62276047) and Shenzhen Science and Technology Program (No. JCYJ20210324121213037).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Li, H., Shao, J., Zeng, X., Xu, H. (2024). Improving Meta-learning for Few-Shot Text Classification via Label Propagation. In: Bifet, A., Davis, J., Krilavičius, T., Kull, M., Ntoutsi, E., Žliobaitė, I. (eds) Machine Learning and Knowledge Discovery in Databases. Research Track. ECML PKDD 2024. Lecture Notes in Computer Science(), vol 14945. Springer, Cham. https://doi.org/10.1007/978-3-031-70362-1_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-70362-1_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-70361-4
Online ISBN: 978-3-031-70362-1
eBook Packages: Computer ScienceComputer Science (R0)
