Abstract
Nowadays deep networks provide excellent results in the context of object segmentation. Available models have been trained on common objects and are not designed to segment specific objects such as fruits or vegetables. In order to help breeders to accelerate and to modernize the process of agriculture products phenotyping, it is necessary to fine tune general models on specific species. Nevertheless, a minimum amount of annotations are required for this retraining step. In this paper, we propose a solution to minimize the annotation workload for each specie. The main idea consists in leveraging the annotations of one specie A in order to fine tune a model on a specie B with few annotations. For this purpose, we propose an Instance-based CycleGAN (ICG) that creates synthetic images of specie B along with corresponding annotations. By fine tuning a segmentation network with these synthetic images and annotations, we show that this network can obtain very good performance on the new specie B, without requiring to manually annotate a large amount of images for this specific specie B.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Li, L., Zhang, Q., Huang, D.: A review of imaging techniques for plant phenotyping. Sensors 14(11), 20078–20111 (2014)
Li, Z., Guo, R., Li, M., Chen, Y., Li, G.: A review of computer vision technologies for plant phenotyping. Comput. Electron. Agric. 176, 105672 (2020)
Dreccer, M.F., Molero, G., Rivera-Amado, C., John-Bejai, C., Wilson, Z.: Yielding to the image: how phenotyping reproductive growth can assist crop improvement and production. Plant Sci. 282, 73–82 (2019). The 4th International Plant Phenotyping Symposium
Goodfellow, I., et al.: Generative adversarial nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27. Curran Associates, Inc. (2014)
Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967–5976 (2017)
Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242–2251 (2017)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988 (2017)
Nazeri, K., Ng, E., Ebrahimi, M.: Image colorization using generative adversarial networks. In: Perales, F.J., Kittler, J. (eds.) AMDO 2018. LNCS, vol. 10945, pp. 85–94. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-94544-6_9
Bulat, A., Tzimiropoulos, G.: Super-fan: integrated facial landmark localization and super-resolution of real-world low resolution faces in arbitrary poses with GANs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
Yu, J., Lin, Z., Yang, J., Shen, X., Xin, L., Huang, T.: Free-form image inpainting with gated convolution. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4470–4479 (2019)
Alqahtani, H., Kavakli-Thorne, M., Kumar, G.: Applications of generative adversarial networks (GANs): an updated review. Arch. Comput. Methods Eng. 28(2), 525–552 (2021)
Yang, H., et al.: Unpaired brain MR-to-CT synthesis using a structure-constrained CycleGAN. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 174–182. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_20
Chen, V., Wu, J., Luo, J.: Group2 SecleGAN: improvement of the CycleGAN with segmentation. Technical report, University of California San Diego (2021). http://noiselab.ucsd.edu/ECE228-2021/projects/report/2Report.pdf. Accessed 1 Nov 2022
Lauenburg, L., et al.: Instance segmentation of unlabeled modalities via cyclic segmentation GAN (2022)
Mo, S., Cho, M., Shin, J.: InstaGAN: instance-aware image-to-image translation. In: 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, 6–9 May 2019. OpenReview.net (2019)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of the 28th International Conference on Neural Information Processing Systems, NIPS 2015, Cambridge, MA, USA, vol. 1, pp. 91–99. MIT Press (2015)
Wu, Y., Kirillov, A., Massa, F., Lo, W.-Y., Girshick, R.: Detectron2 (2019). https://github.com/facebookresearch/detectron2
Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollar, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2017)
Kirillov, A., Girshick, R., He, K., Dollar, P.: Panoptic feature pyramid networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Güler, R.A., Neverova, N., Kokkinos, I.: DensePose: dense human pose estimation in the wild. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7297–7306 (2018)
Abdallah, A.B., Kallel, A., Dammak, M., Ali, A.B.: Olive tree and shadow instance segmentation based on detectron2. In: 2022 6th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), pp. 1–5 (2022)
He, Z.: CycleGAN-TensorFlow-2 (2017). https://github.com/LynnHo/CycleGAN-Tensorflow-2
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)
Rezatofighi, H., Tsoi, N., Gwak, J.Y., Sadeghian, A., Reid, I., Savarese, S.: Generalized intersection over union: a metric and a loss for bounding box regression. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 658–666 (2019)
Chicco, D., Jurman, G.: The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genom. 21(1), 6 (2020)
Sasaki, Y.: The truth of the F-measure. Teach Tutor Mater (2007)
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Díaz Estrada, D.N., Robert, O., Kresovic, M., Torres, C., Muselet, D., Tremeau, A. (2025). Instance-Based CycleGAN for Object Segmentation with Few Annotations. In: Schettini, R., Trémeau, A., Tominaga, S., Bianco, S., Buzzelli, M. (eds) Computational Color Imaging. CCIW 2024. Lecture Notes in Computer Science, vol 15193. Springer, Cham. https://doi.org/10.1007/978-3-031-72845-7_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-72845-7_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72844-0
Online ISBN: 978-3-031-72845-7
eBook Packages: Computer ScienceComputer Science (R0)