Skip to main content
SpringerLink
Log in

Few-Shot Object Detection with Weight Imprinting

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

The goal of few-shot learning is to learn a solution to a problem from limited training samples. In recent years, with the promotion and application of deep neural network–based vision algorithms, the problem of data scarcity has become increasingly prominent. This has prompted comprehensive study on few-shot learning algorithms among academic and industrial communities. This paper first analyzes the bias phenomenon of proposal estimation in the classic transfer learning few-shot object detection paradigm, and then proposes an improved scheme that combines weight imprinting and model decoupling. On the one hand, we extend the weight imprinting algorithm on the general Faster R-CNN framework to enhance the fine-tuning performance; on the other hand, we exploit model decoupling to minimize the over-fitting in data-scarce scenarios. Our proposed method achieves 12.3, 15.0, and 18.9 (nAP) top accuracy on novel set of COCO under 5-shot, 10-shot, and 30-shot settings, and achieves 57.7 and 60.2 (nAP50) top accuracy on novel set of VOC Split 3 under 5-shot and 10-shot settings. Compared with the latest published studies, our proposed method provides a competitive detection performance on novel categories only via fine-tuning. Moreover, it retains the original architecture of the network and is practical in real industrial scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data Availability

The data and programs used in this study are available for public after the article is published.

References

  1. Wang X, Huang TE, Darrell T, Gonzalez JE, Yu F. Frustratingly simple few-shot object detection. In: Proceedings of the 37th International Conference on Machine Learning. 2020 Jul 13. p. 9919–28.

  2. Han G, Huang S, Ma J, He Y, Chang SF. Meta faster r-cnn: towards accurate few-shot object detection with attentive feature alignment. In: Proceedings of the AAAI Conference on Artificial Intelligence. 2022 Jun 28. Vol. 36, No. 1, p. 780–9.

  3. Hsieh TI, Lo YC, Chen HT, Liu TL. One-shot object detection with co-attention and co-excitation. Adv Neural Inf Process Syst. 2019;32.

  4. Chen X, Jiang M, Zhao Q. Leveraging bottom-up and top-down attention for few-shot object detection. arXiv preprint; 2020 Jul 23. arXiv:2007.12104.

  5. Yan X, Chen Z, Xu A, Wang X, Liang X, Lin L. Meta r-cnn: towards general solver for instance-level low-shot learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019. p. 9577–86.

  6. Yang S, Liu L, Xu M. Free lunch for few-shot learning: distribution calibration. arXiv preprint; 2021 Jan 16. arXiv:2101.06395.

  7. Qiao L, Zhao Y, Li Z, Qiu X, Wu J, Zhang C. Defrcn: decoupled faster r-cnn for few-shot object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021. p. 8681–90.

  8. Chen WY, Liu YC, Kira Z, Wang YC, Huang JB. A closer look at few-shot classification. In: International Conference on Learning Representations. 2019 May.

  9. Wang Y, Yao Q, Kwok JT, Ni LM. Generalizing from a few examples: a survey on few-shot learning. ACM Comput Surv (csur). 2020 Jun 12;53(3):1–34.

  10. Wang YX, Ramanan D, Hebert M. Meta-learning to detect rare objects. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019. pp. 9925–34.

  11. Fan Z, Ma Y, Li Z, Sun J. Generalized few-shot object detection without forgetting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021. pp. 4527–36.

  12. Qi H, Brown M, Lowe DG. Low-shot learning with imprinted weights. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2018. pp. 5822–30.

  13. Antonelli S, Avola D, Cinque L, Crisostomi D, Foresti GL, Galasso F, Marini MR, Mecca A, Pannone D. Few-shot object detection: a survey. ACM Comput Surv (CSUR). 2022;54(11s):1–37.

  14. Wang YX, Hebert M. Learning to learn: model regression networks for easy small sample learning. In: European Conference on Computer Vision. Cham: Springer; 2016 Oct 8. pp. 616–34.

  15. Finn C, Abbeel P, Levine S. Model-agnostic meta-learning for fast adaptation of deep networks. In: International conference on machine learning. PMLR; 2017 Jul 17. p. 1126–35.

  16. Kang B, Liu Z, Wang X, Yu F, Feng J, Darrell T. Few-shot object detection via feature reweighting. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019. p. 8420–9.

  17. Xiao Y, Marlet R. Few-shot object detection and viewpoint estimation for objects in the wild. In: European conference on computer vision. Cham: Springer; 2020 Aug 23. p. 192–210.

  18. Liu L, Ma B, Zhang Y, Yi X, Li H. Afd-net: Adaptive fully-dual network for few-shot object detection. In: Proceedings of the 29th ACM International Conference on Multimedia. 2021 Oct 17. p. 2549–57.

  19. Fan Q, Zhuo W, Tang CK, Tai YW. Few-shot object detection with attention-RPN and multi-relation detector. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. p. 4013–22.

  20. Li B, Yang B, Liu C, Liu F, Ji R, Ye Q. Beyond max-margin: Class margin equilibrium for few-shot object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021. p. 7363–72.

  21. Zhang G, Luo Z, Cui K, Lu S. Meta-detr: few-shot object detection via unified image-level meta-learning. arXiv preprint. 2021 Mar 22;2(6). arXiv:2103.11731.

  22. Lin W, Deng Y, Gao Y, Wang N, Zhou J, Liu L, Zhang L, Wang P. CAT: cross-attention transformer for one-shot object detection. arXiv preprint; 2021 Apr 30. arXiv:2104.14984.

  23. Chen H, Wang Y, Wang G, Qiao Y. Lstd: a low-shot transfer detector for object detection. In: Proceedings of the AAAI conference on artificial intelligence. 2018 Apr 29. Vol. 32, No. 1.

    Article  Google Scholar 

  24. Zhang W, Wang YX, Forsyth DA. Cooperating RPN’s improve few-shot object detection. arXiv preprint; 2020 Nov 19. arXiv:2011.10142.

  25. Wu J, Liu S, Huang D, Wang Y. Multi-scale positive sample refinement for few-shot object detection. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVI 16. Springer International Publishing; 2020. pp. 456–72.

  26. Sun B, Li B, Cai S, Yuan Y, Zhang C. Fsce: Few-shot object detection via contrastive proposal encoding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021. pp. 7352–62.

  27. Zhang W, Wang YX. Hallucination improves few-shot object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021. pp. 13008–17.

  28. Wu A, Han Y, Zhu L, Yang Y. Universal-prototype enhancing for few-shot object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021. p. 9567–76.

  29. Khandelwal S, Goyal R, Sigal L. Unit: Unified knowledge transfer for any-shot object detection and segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021. pp. 5951–61.

  30. Gidaris S, Komodakis N. Dynamic few-shot visual learning without forgetting. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2018. pp. 4367–75.

  31. Dhillon GS, Chaudhari P, Ravichandran A, Soatto S. A baseline for few-shot image classification. arXiv preprint; 2019 Sep 6. arXiv:1909.02729.

  32. Jung HG, Lee SW. Few-shot learning with geometric constraints. IEEE Trans Neural Netw Learn Syst. 2020 Jan 1;31(11):4660–72.

  33. Ren S, He K, Girshick R, Sun J. Faster r-cnn: towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst. 2015;28.

  34. Yosinski J, Clune J, Bengio Y, Lipson H. How transferable are features in deep neural networks? Adv Neural Inf Process Syst. 2014;27.

  35. Guirguis K, Hendawy A, Eskandar G, Abdelsamad M, Kayser M, Beyerer J. CFA: constraint-based finetuning approach for generalized few-shot object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022. p. 4039–49.

Download references

Funding

This study was funded by the ChinaTelecom Ideal company.

Author information

Authors and Affiliations

  1. Department of AI Research, Lixiang, ChinaTelecom, No. 189 West Xiuyan Road, Shanghai, China

    Dingtian Yan, Jitao Huang, Hai Sun & Fuqiang Ding

Authors
  1. Dingtian Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jitao Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hai Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fuqiang Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

This work was carried out in close collaboration among all authors. Dingtian Yan and Jitao Huang have conceived the idea, developed the method and experiments, analyzed the obtained data, and wrote the manuscript. Sun Hai and Fuqiang Ding edited the manuscript. All authors have contributed to, seen, and approved the paper.

Corresponding author

Correspondence to Dingtian Yan.

Ethics declarations

Ethics Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed Consent

As this article does not contain any studies with human participants or animals, the informed consent is not applicable.

Conflict of Interest

All authors are members of computer vision lab in ChinaTelecom Lixiang company. The authors declare that this research was conducted in the absence of any financial relationships that could be construed as a potential conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yan, D., Huang, J., Sun, H. et al. Few-Shot Object Detection with Weight Imprinting. Cogn Comput 15, 1725–1735 (2023). https://doi.org/10.1007/s12559-023-10152-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-023-10152-5

Keywords

Search

Navigation