Abstract
In recent years, there has been a worldwide outbreak of coronaviruses, and people are wearing facial masks more and more often. In many cases, people wear masks even when taking photos of themselves, and when photos with the lower half of the face hidden are uploaded to web pages or social networking sites, it is difficult to convey the attractiveness of the photographed persons. In this study, we propose a method to complete the masked region in a face using StyleGAN2, a kind of Generative Adversarial Networks (GAN). In the proposed method, we prepare an image of the same person who is not wearing a mask, and change the orientation and contour of the face of the person in the image to match those of the target image using StyleGAN2. Then, the image with the changed orientation is combined with the target image in which the person is wearing the mask to produce an image in which the mask region is completed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ballester, C., Bertalmío, M., Caselles, V., Sapiro, G., Verdera, J.: Filling-in by joint interpolation of vector fields and gray levels. IEEE Trans. Image Process. 10(8), 1200–1211 (2001)
Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.B.: Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28, 24:1–24:11 (2009)
Bertalmío, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceedings of SIGGRAPH, pp. 417–424 (2000)
Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)
Din, N.U., Javed, K., Bae, S., Yi, J.: A novel GAN-based network for unmasking of masked face. IEEE Access 8, 44276–44287 (2020)
Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: Proceedings of IEEE International Conference on Computer Vision, vol. 2, pp. 1033–1038 (1999)
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 4401–4410 (2019)
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of StyleGAN. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 8110–8119 (2020)
Kawai, N., Sato, T., Yokoya, N.: Image inpainting considering brightness change and spatial locality of textures and its evaluation. In: Proceedings of Pacific-Rim Symposium on Image and Video Technology, pp. 271–282 (2009)
Kawai, N., Yokoya, N.: Image inpainting considering symmetric patterns. In: Proceedings of IAPR International Conference on Pattern Recognition, pp. 2744–2747 (2012)
King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)
Li, Y., Liu, S., Yang, J., Yang, M.H.: Generative face completion, pp. 5892–5900 (2017)
Nazeri, K., Ng, E., Joseph, T., Qureshi, F.Z., Ebrahimi, M.: EdgeConnect: Structure guided image inpainting using edge prediction, pp. 3265–3274 (2019)
Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. 22(3), 313–318 (2003)
Yang, Y., Guo, X.: Generative landmark guided face inpainting. In: Proceedings of Chinese Conference on Pattern Recognition and Computer Vision, pp. 14–26 (2020)
Acknowledgements
This work was supported by JSPS KAKENHI Grant Numbers JP18H03273, JP18H04116, JP21H03483.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Koike, H., Kawai, N. (2022). Facial Mask Region Completion Using StyleGAN2 with a Substitute Face of the Same Person. In: Sumi, K., Na, I.S., Kaneko, N. (eds) Frontiers of Computer Vision. IW-FCV 2022. Communications in Computer and Information Science, vol 1578. Springer, Cham. https://doi.org/10.1007/978-3-031-06381-7_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-06381-7_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06380-0
Online ISBN: 978-3-031-06381-7
eBook Packages: Computer ScienceComputer Science (R0)