Abstract
Deep learning models are the state-of-the-art approach to deal with semantic segmentation tasks. However, training deep models require a considerable amount of images that might be difficult to obtain. This issue can be faced by means of data augmentation techniques that generate new images by applying geometric or colour transformations, or more recently by mixing several images using techniques such as CutMix or CarveMix. Unfortunately, mixing strategies are usually implemented as ad-hoc methods and are difficult to incorporate into the pipeline to train segmentation models. In this work, we present a library that implements several mixing strategies for data augmentation in semantic segmentation tasks. In particular, we provide a set of callbacks that can be integrated into the training pipeline of FastAI segmentation models. We have tested the library with a vineyard dataset and show the benefits of combining mixing strategies with traditional data augmentation techniques; namely an improvement of almost 5% was achieved using these methods regarding models trained only with traditional data augmentation methods.
This work was partially supported by Ministerio de Ciencia e Innovación [PID2020-115225RB-I00/AEI/10.13039/501100011033].
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Buslaev, A., Iglovikov, V.I., Khvedchenya, E., Parinov, A., Druzhinin, M., Kalinin, A.A.: Albumentations: fast and flexible image augmentations. Information 11(2), 125 (2020)
DeVries, T., Taylor, G.W.: Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552 (2017)
Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., Martinez-Gonzalez, P., Garcia-Rodriguez, J.: A survey on deep learning techniques for image and video semantic segmentation. Appl. Soft Comput. 70, 41–65 (2018)
Hao, S., Zhou, Y., Guo, Y.: A brief survey on semantic segmentation with deep learning. Neurocomputing 406, 302–321 (2020)
Howard, J., Gugger, S.: Deep Learning for Coders with fastai and PyTorch. O’Reilly Media, Sebastopol (2020)
Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18(2), 203–211 (2021)
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Marani, R., Milella, A., Petitti, A., Reina, G.: Deep neural networks for grape bunch segmentation in natural images from a consumer-grade camera. Precision Agric. 22(2), 387–413 (2021)
Opitz, J., Burst, S.: Macro f1 and macro f1. arXiv preprint arXiv:1911.03347 (2019)
Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Qin, J., Fang, J., Zhang, Q., Liu, W., Wang, X., Wang, X.: Resizemix: mixing data with preserved object information and true labels. arXiv preprint arXiv:2012.11101 (2020)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Salimans, T., et al.: Improved techniques for training GANs. In: 30th International Conference on Neural Information Processing Systems, pp. 2234–2242. Curran Associates Inc. (2016)
Simard, P., Victorri, B., LeCun, Y., Denker, J.S.: Tangent prop-a formalism for specifying selected invariances in an adaptive network. In: In Proceedings of Neural Information Processing Systems (NeuriPS 1991), vol. 91, pp. 895–903 (1991)
Singh, K.K., Yu, H., Sarmasi, A., Pradeep, G., Lee, Y.J.: Hide-and-seek: a data augmentation technique for weakly-supervised localization and beyond. arXiv preprint arXiv:1811.02545 (2018)
Takahashi, R., Matsubara, T., Uehara, K.: RICAP: random image cropping and patching data augmentation for deep CNNs. In: Asian Conference on Machine Learning, pp. 786–798. PMLR (2018)
Yun, S., Han, D., Oh, S.J., Chun, S., Choe, J., Yoo, Y.: CutMix: regularization strategy to train strong classifiers with localizable features. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6023–6032 (2019)
Zeng, Q., Ma, X., Cheng, B., Zhou, E., Pang, W.: GANs-based data augmentation for citrus disease severity detection using deep learning. IEEE Access 8, 172882–172891 (2020)
Zhang, X., et al.: CarveMix: a simple data augmentation method for brain lesion segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 196–205. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_19
Zhu, Q., Wang, Y., Yin, L., Yang, J., Liao, F., Li, S.: Selfmix: a self-adaptive data augmentation method for lesion segmentation. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) MICCAI 2022. LNCS, vol. 13434, pp. 683–692. Springer, Cham (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Escobedo, R., Heras, J. (2023). Mixing Data Augmentation Methods for Semantic Segmentation. In: Dorronsoro, B., Chicano, F., Danoy, G., Talbi, EG. (eds) Optimization and Learning. OLA 2023. Communications in Computer and Information Science, vol 1824. Springer, Cham. https://doi.org/10.1007/978-3-031-34020-8_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-34020-8_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-34019-2
Online ISBN: 978-3-031-34020-8
eBook Packages: Computer ScienceComputer Science (R0)