Skip to main content
SpringerLink
Log in

Bias oriented unbiased data augmentation for cross-bias representation learning

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

The biased cues in the training data may build strong connections between specific targets and unexpected concepts, leading the learned representations could not be applied to real-world data that does not contain the same biased cues. To learn cross-bias representations which can generalize on unbiased datasets by only using biased data, researchers focus on reducing the influence of biased cues through unbiased sampling or augmentation on the basis of artificial experience. However, the distributions of biased cues in the dataset are neglected, which limits the performance of these methods. In this paper, we propose a bias oriented data augmentation to enhance the cross-bias generalization by enlarging “safety” and “unbiasedness” constraints in the training data without manual prior intervention. The safety constraint is proposed to maintain the class-specific information for augmentation while the unbiasedness constraint reduces the statistical correlation of bias information and class labels. Experiments under different biased proportions on four synthetic/real-world datasets show that the proposed approach could improve the performance of other SOTA debiasing approaches (colored MNIST: 0.35–26.14%, corrupted CIFAR10: 3.14–8.44%, BFFHQ: 1.50% and BAR: 1.72%).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Gaur, L., Bhatia, U., Jhanjhi, N., Muhammad, G., Masud, M.: Medical image-based detection of covid-19 using deep convolution neural networks. Multimed. Syst. (2021). https://doi.org/10.1007/s00530-021-00794-6

    Article  Google Scholar 

  2. Wei, P., Wang, B.: Food image classification and image retrieval based on visual features and machine learning. Multimed. Syst. (2020). https://doi.org/10.1007/s00530-020-00673-6

    Article  Google Scholar 

  3. Tayal, A., Gupta, J., Solanki, A., Bisht, K., Nayyar, A., Masud, M.: Dl-cnn-based approach with image processing techniques for diagnosis of retinal diseases. Multimed. Syst. 28, 1417–1438 (2021). https://doi.org/10.1007/s00530-021-00769-7

    Article  Google Scholar 

  4. Ta, N., Chen, H., Lyu, Y., Wu, T.: Ble-net: boundary learning and enhancement network for polyp segmentation. Multimed. Syst. (2022). https://doi.org/10.1007/s00530-022-00900-2

    Article  Google Scholar 

  5. Xia, K., Gu, X., Zhang, Y.: Oriented grouping-constrained spectral clustering for medical imaging segmentation. Multimed. Syst. 26(1), 27–36 (2020)

    Article  Google Scholar 

  6. Olimov, B., Sanjar, K., Din, S., Ahmad, A., Paul, A., Kim, J.: Fu-net: fast biomedical image segmentation model based on bottleneck convolution layers. Multimed. Syst. 27(4), 637–650 (2021)

    Article  Google Scholar 

  7. Poongodi, M., Hamdi, M., Wang, H.: Image and audio caps: automated captioning of background sounds and images using deep learning. Multimed. Syst. (2022). https://doi.org/10.1007/s00530-022-00902-0

    Article  Google Scholar 

  8. Xu, N., Liu, A.-A., Nie, W., Su, Y.: Multi-guiding long short-term memory for video captioning. Multimed. Syst. 25(6), 663–672 (2019)

    Article  Google Scholar 

  9. Shen, Z., Cui, P., Zhang, T., Kunag, K.: Stable learning via sample reweighting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 5692–5699 (2020)

  10. Nam, J., Cha, H., Ahn, S.-S., Lee, J., Shin, J.: Learning from failure: de-biasing classifier from biased classifier. Adv. Neural Inf. Process. Syst. 33, 20673–20684 (2020)

    Google Scholar 

  11. Bahng, H., Chun, S., Yun, S., Choo, J., Oh, S.J.: Learning de-biased representations with biased representations. In: International Conference on Machine Learning, pp. 528–539. PMLR (2020)

  12. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  13. Bai, H., Sun, R., Hong, L., Zhou, F., Ye, N., Ye, H.-J., Chan, S.-H.G., Li, Z.: Decaug: out-of-distribution generalization via decomposed feature representation and semantic augmentation. arXiv preprint arXiv:2012.09382 (2020)

  14. Kim, B., Kim, H., Kim, K., Kim, S., Kim, J.: Learning not to learn: training deep neural networks with biased data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9012–9020 (2019)

  15. Tartaglione, E., Barbano, C.A., Grangetto, M.: End: entangling and disentangling deep representations for bias correction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13508–13517 (2021)

  16. Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F.A., Brendel, W.: Imagenet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In: International Conference on Learning Representations (2019). https://openreview.net/forum?id=Bygh9j09KX

  17. Niu, Y., Tang, K., Zhang, H., Lu, Z., Hua, X.-S., Wen, J.-R.: Counterfactual vqa: a cause-effect look at language bias. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12700–12710 (2021)

  18. Li, Y., Vasconcelos, N.: Repair: removing representation bias by dataset resampling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9572–9581 (2019)

  19. Zhang, X., Cui, P., Xu, R., Zhou, L., He, Y., Shen, Z.: Deep stable learning for out-of-distribution generalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5372–5382 (2021)

  20. Li, L., Gao, K., Cao, J., Huang, Z., Weng, Y., Mi, X., Yu, Z., Li, X., Xia, B.: Progressive domain expansion network for single domain generalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 224–233 (2021)

  21. Wang, H., He, Z., Lipton, Z.L., Xing, E.P.: Learning robust representations by projecting superficial statistics out. In: International Conference on Learning Representations (2019). https://openreview.net/forum?id=rJEjjoR9K7

  22. Cadene, R., Dancette, C., Cord, M., Parikh, D., et al.: Rubi: reducing unimodal biases for visual question answering. Adv. Neural. Inf. Process. Syst. 32, 841–852 (2019)

    Google Scholar 

  23. Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6(1), 1–48 (2019)

    Article  Google Scholar 

  24. Summers, C., Dinneen, M.J.: Improved mixed-example data augmentation. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1262–1270 (2019). IEEE

  25. Kang, G., Dong, X., Zheng, L., Yang, Y.: Patchshuffle regularization. arXiv preprint arXiv:1707.07103 (2017)

  26. Takahashi, R., Matsubara, T., Uehara, K.: Data augmentation using random image cropping and patching for deep cnns. IEEE Trans. Circuits Syst. Video Technol. 30(9), 2917–2931 (2019)

    Article  Google Scholar 

  27. DeVries, T., Taylor, G.W.: Dataset augmentation in feature space. arXiv preprint arXiv:1702.05538 (2017)

  28. Doersch, C.: Tutorial on variational autoencoders. arXiv preprint arXiv:1606.05908 (2016)

  29. Bowles, C., Chen, L., Guerrero, R., Bentley, P., Gunn, R., Hammers, A., Dickie, D.A., Hernández, M.V., Wardlaw, J., Rueckert, D.: Gan augmentation: augmenting training data using generative adversarial networks. arXiv preprint arXiv:1810.10863 (2018)

  30. Kortylewski, A., Egger, B., Schneider, A., Gerig, T., Vetter, T.: Analyzing and reducing the damage of dataset bias to face recognition with synthetic data. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2019)

  31. Jaipuria, N., Zhang, X., Bhasin, R., Arafa, M., Chakravarty, P., Shrivastava, S., Manglani, S., Murali, V.N.: Deflating dataset bias using synthetic data augmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp 772–773 (2020)

  32. Qian, X., Fu, Y., Tao, X., Wang, W., Xue, X.: Pose-normalized image generation for person re-identification. In: 15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, part ix. European Conference on Computer Vision (2018)

  33. Zhang, X., Tseng, N., Syed, A., Bhasin, R., Jaipuria, N.: Simbar: Single image-based scene relighting for effective data augmentation for automated driving vision tasks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3718–3728 (2022)

  34. Mao, X., Li, Q., Xie, H., Lau, R., Smolley, S.P.: Least squares generative adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV) (2017)

  35. Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 214–223. PMLR (2017). https://proceedings.mlr.press/v70/arjovsky17a.html

  36. Zhang, Z., Sabuncu, M.: Generalized cross entropy loss for training deep neural networks with noisy labels. Adv. Neural Inf. Process. Syst. 31, 8778–8788 (2018)

    Google Scholar 

  37. Lee, J., Kim, E., Lee, J., Lee, J., Choo, J.: Learning debiased representation via disentangled feature augmentation. Adv. Neural Inf. Process. Syst. 34, 25123–25133 (2021)

    Google Scholar 

  38. Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images. Technical Report 0, University of Toronto, Toronto, Ontario (2009)

  39. Hendrycks, D., Dietterich, T.: Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261 (2019)

  40. Kim, E., Lee, J., Choo, J.: Biaswap: Removing dataset bias with bias-tailored swapping augmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14992–15001 (2021)

  41. Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4401–4410 (2019)

  42. Wang, H., He, Z., Lipton, Z.C., Xing, E.P.: Learning robust representations by projecting superficial statistics out. arXiv preprint arXiv:1903.06256 (2019)

  43. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

Download references

Author information

Authors and Affiliations

  1. Institute of Computing Technology, Chinese Academy of Science, Beijing, 100080, China

    Lei Li, Fan Tang, Juan Cao & Danding Wang

  2. University of Chinese Academy of Sciences, Beijing, 100049, China

    Lei Li, Fan Tang, Juan Cao & Danding Wang

  3. Renmin University of China, Beijing, 100872, China

    Xirong Li

Authors
  1. Lei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fan Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Juan Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xirong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Danding Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fan Tang.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, L., Tang, F., Cao, J. et al. Bias oriented unbiased data augmentation for cross-bias representation learning. Multimedia Systems 29, 725–738 (2023). https://doi.org/10.1007/s00530-022-01013-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-022-01013-6

Keywords

Search

Navigation