Abstract
It has been increasingly recognized that meta-learning-based approaches provide a promising way to handle challenges to few-shot learning. In this paper, we incorporate the channel attention in the main framework of simple-CNAPS proposed by Bateni et al. to develop a model more appropriate for few-shot image classification. In detail, we replace FiLM layers in simple-CNAPS with channel attention blocks which scale the image channels according to the relationship between task information and feature maps rather than only the task information. This replacement makes the feature extractor more expressive. Moreover, it allows us to take the interaction of different image channels into account. In addition, to alleviate the computational bias caused by small sample size, we provide a method to estimate class centers with perturbations. Finally, the effectiveness of the model is verified by experiments on the few-shot image classification benchmark datasets.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: 27th International Conference on Neural Information Processing Systems (NeurIPS), pp. 3104–3112. MIT, Montreal (2014)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778. IEEE, Las Vegas (2016)
Chen, M., Zhang, Y., Qiu, M., Guizani, N., Hao, Y.: SPHA: smart personal health advisor based on deep analytics. IEEE Commun. Mag. 56(3), 164–169 (2018)
Ren, M., Triantafillou, E., Ravi, S., Snell, J., et al.: Meta learning for semi-supervised few-shot classification. CoRR, abs/1803.00676(2018)
Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639), 115–118 (2017)
Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: 34th International Conference on Machine Learning, pp. 1126–1135. JMLR, Sydney (2017)
Hospedales, T., Antoniou, A., Micaelli, P., Storkey, A.: Meta-learning in neural networks: a survey. arXiv preprint arXiv:2004.05439 (2020)
Rajeswaran, A., Finn, C., Kakade, S.M., Levine, S.: Meta-learning with implicit gradients. In: 33rd Conference on Neural Information Processing Systems (NeurIPS), pp. 113–124. MIT, Vancouver (2019)
Nichol, A., Achiam, J., Schulman, J.: On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999 (2018)
Ravi, S., Larochelle, H.: Optimization as a model for few shot learning. In: 5th International Conference on Learning Representations (ICLR), OpenReview.net, Toulon (2016)
Mishra, N., Rohaninejad, M., Chen, X., Abbeel, P.: A simple neural attentive meta-learner. In: 6th International Conference on Learning Representations (ICLR), OpenReview.net, Vancouver (2018)
Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al.: Matching networks for one shot learning. In: 29th Annual Conference on Neural Information Processing Systems (NeurIPS), pp. 3630–3638. MIT, Barcelona (2016)
Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: 30rd Annual Conference on Neural Information Processing Systems (NeurIPS), Long Beach, USA, pp. 4077–4087 (2017)
Tian, Y., Wang, Y., Krishnan, D., Tenenbaum, J.B., Isola, P.: Rethinking few-shot image classification: a good embedding is all you need? In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12359, pp. 266–282. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58568-6_16
Requeima, J., Gordon, J., Bronskill, J., Nowozin, S., Turner, R.E.: Fast and flexible multi-task classification using conditional neural adaptive processes. In: 32rd Conference on Neural Information Processing Systems(NeurIPS), Vancouver, Canada, pp. 7957–7968 (2019)
Perez, E., Strub, F., Vries, H.D., Dumoulin, V., Courville, A.: FiLM: visual reasoning with a general conditioning layer. In: 32nd AAAI Conference on Artificial Intelligence (AAAI), pp. 3942–3951. AAAI press, New Orleans (2018)
Bateni, P., Goyal, R., Masrani, V., Wood, F., Sigal, L.: Improved few-shot visual classification. In: 2020 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14481–14490. IEEE, Seattle (2020)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7132–7141. IEEE, Salt Lake City (2018)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Chen, J., Wu, X., Li, Y., Li, Q., Zhan, L., Chung, F.: A closer look at the training strategy for modern meta-learning. In: 33th International Conference on Neural Information Processing Systems (NeurIPS). Virtual (2020)
Chen, Y., et al.: Modular meta-learning with shrinkage. In: 33th International Conference on Neural Information Processing Systems (NeurIPS). Virtual (2020)
Lu, J., Gong, P., Ye, J.: Learning from very few samples: a survey. CoRR: abs/2009.02653 (2020)
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)
Lake, B.M., Salakhutdinov, R., Tenenbaum, J.B.: Human-level concept learning through probabilistic program induction. Science 350(6266), 1332–1338 (2015)
Triantafillou, E., et al.: Meta-dataset: a dataset of datasets for learning to learn from examples. In: 8th International Conference on Learning Representations (ICLR). OpenReview.net, Addis Ababa (2020)
Oreshkin, B., Rodrıguez Lopez, P., Lacoste, A.: TADAM: task dependent adaptive metric for improved few-shot learning. In: 31st Annual Conference on Neural Information Processing Systems (NeurIPS), Montreal, Canada, pp. 721–731 (2018)
Liu, Y., Lee, J., Park, M., Kim, S., Yang, Y.: Transductive propagation network for few-shot learning. CoRR, abs/1805.10002 (2018)
Rusu, A.A., et al.: Meta-learning with latent embedding optimization. CoRR, abs/1807.05960 (2018)
Acknowledgements
This work was partly supported by the National Natural Science Foundation of China under Grant No. 61572002, No. 61690201, and No. 61732001.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, Z., Mu, K. (2021). Integrating Task Information into Few-Shot Classifier by Channel Attention. In: Qiu, H., Zhang, C., Fei, Z., Qiu, M., Kung, SY. (eds) Knowledge Science, Engineering and Management. KSEM 2021. Lecture Notes in Computer Science(), vol 12817. Springer, Cham. https://doi.org/10.1007/978-3-030-82153-1_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-82153-1_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-82152-4
Online ISBN: 978-3-030-82153-1
eBook Packages: Computer ScienceComputer Science (R0)