Skip to main content

Instance-Based CycleGAN for Object Segmentation with Few Annotations

  • Conference paper
  • First Online:
Computational Color Imaging (CCIW 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15193))

Included in the following conference series:

  • 84 Accesses

Abstract

Nowadays deep networks provide excellent results in the context of object segmentation. Available models have been trained on common objects and are not designed to segment specific objects such as fruits or vegetables. In order to help breeders to accelerate and to modernize the process of agriculture products phenotyping, it is necessary to fine tune general models on specific species. Nevertheless, a minimum amount of annotations are required for this retraining step. In this paper, we propose a solution to minimize the annotation workload for each specie. The main idea consists in leveraging the annotations of one specie A in order to fine tune a model on a specie B with few annotations. For this purpose, we propose an Instance-based CycleGAN (ICG) that creates synthetic images of specie B along with corresponding annotations. By fine tuning a segmentation network with these synthetic images and annotations, we show that this network can obtain very good performance on the new specie B, without requiring to manually annotate a large amount of images for this specific specie B.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Li, L., Zhang, Q., Huang, D.: A review of imaging techniques for plant phenotyping. Sensors 14(11), 20078–20111 (2014)

    Article  Google Scholar 

  2. Li, Z., Guo, R., Li, M., Chen, Y., Li, G.: A review of computer vision technologies for plant phenotyping. Comput. Electron. Agric. 176, 105672 (2020)

    Article  Google Scholar 

  3. Dreccer, M.F., Molero, G., Rivera-Amado, C., John-Bejai, C., Wilson, Z.: Yielding to the image: how phenotyping reproductive growth can assist crop improvement and production. Plant Sci. 282, 73–82 (2019). The 4th International Plant Phenotyping Symposium

    Article  Google Scholar 

  4. Goodfellow, I., et al.: Generative adversarial nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27. Curran Associates, Inc. (2014)

    Google Scholar 

  5. Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967–5976 (2017)

    Google Scholar 

  6. Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242–2251 (2017)

    Google Scholar 

  7. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988 (2017)

    Google Scholar 

  8. Nazeri, K., Ng, E., Ebrahimi, M.: Image colorization using generative adversarial networks. In: Perales, F.J., Kittler, J. (eds.) AMDO 2018. LNCS, vol. 10945, pp. 85–94. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-94544-6_9

    Chapter  Google Scholar 

  9. Bulat, A., Tzimiropoulos, G.: Super-fan: integrated facial landmark localization and super-resolution of real-world low resolution faces in arbitrary poses with GANs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

    Google Scholar 

  10. Yu, J., Lin, Z., Yang, J., Shen, X., Xin, L., Huang, T.: Free-form image inpainting with gated convolution. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4470–4479 (2019)

    Google Scholar 

  11. Alqahtani, H., Kavakli-Thorne, M., Kumar, G.: Applications of generative adversarial networks (GANs): an updated review. Arch. Comput. Methods Eng. 28(2), 525–552 (2021)

    Article  MathSciNet  Google Scholar 

  12. Yang, H., et al.: Unpaired brain MR-to-CT synthesis using a structure-constrained CycleGAN. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 174–182. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_20

    Chapter  Google Scholar 

  13. Chen, V., Wu, J., Luo, J.: Group2 SecleGAN: improvement of the CycleGAN with segmentation. Technical report, University of California San Diego (2021). http://noiselab.ucsd.edu/ECE228-2021/projects/report/2Report.pdf. Accessed 1 Nov 2022

  14. Lauenburg, L., et al.: Instance segmentation of unlabeled modalities via cyclic segmentation GAN (2022)

    Google Scholar 

  15. Mo, S., Cho, M., Shin, J.: InstaGAN: instance-aware image-to-image translation. In: 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, 6–9 May 2019. OpenReview.net (2019)

    Google Scholar 

  16. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of the 28th International Conference on Neural Information Processing Systems, NIPS 2015, Cambridge, MA, USA, vol. 1, pp. 91–99. MIT Press (2015)

    Google Scholar 

  17. Wu, Y., Kirillov, A., Massa, F., Lo, W.-Y., Girshick, R.: Detectron2 (2019). https://github.com/facebookresearch/detectron2

  18. Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollar, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2017)

    Google Scholar 

  19. Kirillov, A., Girshick, R., He, K., Dollar, P.: Panoptic feature pyramid networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  20. Güler, R.A., Neverova, N., Kokkinos, I.: DensePose: dense human pose estimation in the wild. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7297–7306 (2018)

    Google Scholar 

  21. Abdallah, A.B., Kallel, A., Dammak, M., Ali, A.B.: Olive tree and shadow instance segmentation based on detectron2. In: 2022 6th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), pp. 1–5 (2022)

    Google Scholar 

  22. He, Z.: CycleGAN-TensorFlow-2 (2017). https://github.com/LynnHo/CycleGAN-Tensorflow-2

  23. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)

    Google Scholar 

  24. Rezatofighi, H., Tsoi, N., Gwak, J.Y., Sadeghian, A., Reid, I., Savarese, S.: Generalized intersection over union: a metric and a loss for bounding box regression. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 658–666 (2019)

    Google Scholar 

  25. Chicco, D., Jurman, G.: The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genom. 21(1), 6 (2020)

    Article  Google Scholar 

  26. Sasaki, Y.: The truth of the F-measure. Teach Tutor Mater (2007)

    Google Scholar 

  27. Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Damien Muselet .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Díaz Estrada, D.N., Robert, O., Kresovic, M., Torres, C., Muselet, D., Tremeau, A. (2025). Instance-Based CycleGAN for Object Segmentation with Few Annotations. In: Schettini, R., Trémeau, A., Tominaga, S., Bianco, S., Buzzelli, M. (eds) Computational Color Imaging. CCIW 2024. Lecture Notes in Computer Science, vol 15193. Springer, Cham. https://doi.org/10.1007/978-3-031-72845-7_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-72845-7_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-72844-0

  • Online ISBN: 978-3-031-72845-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics