Abstract
In this study, we explore recent advances in photorealistic style transfer methods to make visual predictions of outdoor scenes. These methods transfer the elements’ visual appearance from one photo (style image) to another (content image), maintaining the original composition of the elements in the original image. However, the search for reference images containing the same elements as the content image and presenting all the desired style characteristics makes the process challenging and time-consuming. To overcome this challenge, we propose a dynamic search method based on the transient scene attributes performed in a dataset developed especially for this task. Our team developed a set of 924 3D images divided into six scenario groups, with the main elements found in the outdoor scenes to be used as style images. Each group has stylizations of the four seasons, the time of day, the presence or absence of rain, snow, and cloudy skies. In the end, we measured the similarity of the results obtained with real images. The structural similarity index measure (SSIM) reaches an average score greater than 0.8.
Supported by Sidia Institute of Science and Technology.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cambridge in Color - a learning community for photographers (2005–2020). https://www.cambridgeincolour.com. Accessed 12 Dec 2020
Structural similarity. Wikipedia, February 2021. https://en.wikipedia.org/wiki/Structural_similarity. Accessed 12 Feb 2021
Fix, E., HodgesHodges, J.L.: Discriminatory analysis. Nonparametric discrimination: consistency properties. Int. Statist. Rev./Revue Internationale de Statistique 57(3), 238–247 (1989). http://www.jstor.org/stable/1403797
Fraser, B., Schewe, J.: Real World Image Sharpening with Adobe Photoshop, Camera Raw, and Lightroom, 2nd edn. Peachpit Press, Berkeley (2009)
Karacan, L., Akata, Z., Erdem, A., Erdem, E.: Manipulating attributes of natural scenes via hallucination. ACM Trans. Graph. 39, 1–17 (2019)
Laffont, P.Y., Ren, Z., Tao, X., Qian, C., Hays, J.: Transient attributes for high-level understanding and editing of outdoor scenes. ACM Trans. Graph. (Proceedings of SIGGRAPH) 33(4), 1–11 (2014)
Li, Y., Liu, M.Y., Li, X., Yang, M.H., Kautz, J.: A closed-form solution to photorealistic image stylization. In: ECCV (2018)
Min, K., Sim, D.: Confidence-based adaptive frame rate up-conversion. EURASIP J. Adv. Signal Process. 2013, 13 (2013). https://doi.org/10.1186/1687-6180-2013-13
wiredfool, A.C., et al.: Pillow: 3.1.0, January 2016. https://doi.org/10.5281/zenodo.44297
Zhou, B., et al.: Semantic understanding of scenes through the ade20k dataset. Int. J. Comput. Vis. 127, 302–321 (2018)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Fernandes, E., Aleixo, E., Barreira, W.J., Gadelha, M.R., Khurshid, A., Tamayo, S.C. (2021). Visual Prediction Based on Photorealistic Style Transfer. In: Degen, H., Ntoa, S. (eds) Artificial Intelligence in HCI. HCII 2021. Lecture Notes in Computer Science(), vol 12797. Springer, Cham. https://doi.org/10.1007/978-3-030-77772-2_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-77772-2_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77771-5
Online ISBN: 978-3-030-77772-2
eBook Packages: Computer ScienceComputer Science (R0)