Abstract
In many existing AI methods, the reasons behind the decisions made by a trained model are not easy to explain. This often leads to a black-box design that is not interpretable, which makes it a delicate issue to adopt such methods in an application related to safety. We consider generative adversarial networks that are often used to generate data for further use in deep learning applications where not much data is available. In particular, we deal with the StyleGAN approach for generating synthetic observations of swimmers. This paper provides a pipeline that can clearly explain the synthesized images after projecting them to a lower dimensional space. These understood images can later be chosen to train a swimmer safety observation framework. The main goal of our paper is to achieve a higher level of abstraction by which one can explain the variation of synthesized swimmer images in low dimension space. A standard similarity measure is used to evaluate our pipeline and validate a low intra-class variation of established swimmer clusters representing similar swimming style within a low dimensional space.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alaluf, Y., Patashnik, O., Cohen-Or, D.: ReStyle: a residual-based StyleGAN encoder via iterative refinement (2021)
Ballard, A., et al.: Energy landscapes for machine learning. Phys. Chem. 19, 12585–12603 (2017)
Collins, E., Bala, R., Price, B., Süsstrunk, S.: Editing in style: uncovering the local semantics of GANs. arXiv (2020). https://arxiv.org/abs/2004.14367
Goodfellow, I., et al.: Generative adversarial networks. arXiv (2014). https://arxiv.org/abs/1406.2661
Hartigan, J., Wong, M.: A k-means clustering algorithm. JSTOR: Appl. Stat. 28, 100–108 (1979)
Horoi, S., Huang, J., Wolf, G., Krishnaswamy, S.: Visualizing high-dimensional trajectories on the loss-landscape of ANNs (2021)
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006)
Hong, S., et al.: 3D-StyleGAN: a style-based generative adversarial network for generative modeling of three-dimensional medical images. arXiv (2021). https://arxiv.org/abs/2107.09700
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: 2019 IEEE/CVF Conference On Computer Vision And Pattern Recognition (CVPR), pp. 4396–4405 (2019)
Kostic, A., et al.: The dynamics of the human infant gut microbiome in development and in progression toward type 1 diabetes. Cell Host Microbe 20, 121 (2016)
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. CoRR. abs/1812.04948 (2018). https://arxiv.org/abs/1812.04948
Karras, T., et al.: Alias-free generative adversarial networks. arXiv (2021). https://arxiv.org/abs/2106.12423
Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive growing of GANs for improved quality, stability, and variation. arXiv (2017). https://arxiv.org/abs/1710.10196
Krizhevsky, A.: Learning multiple layers of features from tiny images (2009)
Github (2022). https://github.com/datitran/raccoon-dataset/blob/master/generate-tfrecord.py
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of StyleGAN. arXiv (2019). https://arxiv.org/abs/1912.04958
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)
Lee, C., Liu, Z., Wu, L., Luo, P.: MaskGAN: towards diverse and interactive facial image manipulation. In: IEEE Conference on Computer Vision And Pattern Recognition (CVPR) (2020)
Lewis, K., Varadharajan, S., Kemelmacher-Shlizerman, I.: TryOnGAN: body-aware try-on via layered interpolation. arXiv (2021). https://arxiv.org/abs/2101.02285
Linderman, G., Rachh, M., Hoskins, J., Steinerberger, S., Kluger, Y.: Fast interpolation-based t-SNE for improved visualization of single-cell RNA-seq data. Nat. Methods 16, 1 (2019)
Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)
Moon, K., et al.: Visualizing structure and transitions in high-dimensional biological data (2019)
Nguyen, A., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: high confidence predictions for unrecognizable images. In: 2015 IEEE Conference On Computer Vision And Pattern Recognition (CVPR), pp. 427–436 (2015)
Saeed, W., Omlin, C.: Explainable AI (XAI): a systematic meta-survey of current challenges and future opportunities. Knowl.-Based Syst. 263, 110273 (2023)
Szegedy, C., et al.: Intriguing properties of neural networks (2014)
Skorokhodov, I., Tulyakov, S., Elhoseiny, M.: StyleGAN-V: a continuous video generator with the price, image quality and perks of StyleGAN2. arXiv (2021). https://arxiv.org/abs/2112.14683
Samaria, F., Harter, A.: Parameterisation of a stochastic model for human face identification. In: IEEE Workshop On Applications Of Computer Vision - Proceedings, vol. 22, pp. 138–142 (1995)
Song, W., Wang, L., Liu, P., Choo, K.: Improved t-SNE based manifold dimensional reduction for remote sensing data processing. Multimed. Tools Appl. 78, 4311–4326 (2019)
Wu, J., Wang, J., Xiao, H., Ling, J.: Visualization of high dimensional turbulence simulation data using t-SNE (2017)
Zhang, R.: Making convolutional networks shift-invariant again. arXiv (2019). https://arxiv.org/abs/1904.11486
Acknowledgement
The authors acknowledge partial funding of their work by Bundesministerium für Digitales und Verkehr (BMDV) within the project RescueFly as well as by Bundesministerium für Bildung und Forschung (BMBF) within the project KI@MINT.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mansouri Yarahmadi, A., Breuß, M., Khan Mohammadi, M. (2023). Explaining StyleGAN Synthesized Swimmer Images in Low-Dimensional Space. In: Tsapatsoulis, N., et al. Computer Analysis of Images and Patterns. CAIP 2023. Lecture Notes in Computer Science, vol 14184. Springer, Cham. https://doi.org/10.1007/978-3-031-44237-7_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-44237-7_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44236-0
Online ISBN: 978-3-031-44237-7
eBook Packages: Computer ScienceComputer Science (R0)