Abstract
Blind face restoration, as a kind of face restoration method dealing with complex degradation, has been a challenging research hotspot recently. However, due to the influence of a variety of degradation in low-quality images, artifacts commonly exist in the low fidelity results of existing methods, resulting in a lack of natural and realistic texture details. In this paper, we propose a degradation-aware blind face restoration method based on a high-quality vector quantization (VQ) codebook to improve the degradation-aware capability and texture quality. The overall framework consists of Degradation-aware Module (DAM), Texture Refinement Module (TRM) and Global Restoration Module (GRM). DAM adopts the channel attention mechanism to adjust the weight of feature components in different channels, so that it has the ability to perceive complex degradation from redundant information. In TRM, continuous vectors are quantized and replaced with high-quality discretized vectors in the VQ codebook to add texture details. GRM adopts the reverse diffusion process of the pre-trained diffusion model to restore the image globally. Experiments show that our method outperforms state-of-the-art methods on synthetic and real-world datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, C., Li, X., Yang, L., Lin, X., Zhang, L., Wong, K.Y.K.: Progressive semantic-aware style transformation for blind face restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11896–11905 (2021)
Conde, M.V., Choi, U.J., Burchi, M., Timofte, R.: Swin2sr: swinv2 transformer for compressed image super-resolution and restoration. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds.) ECCV 2022. LNCS, vol. 13802, pp. 669–687. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-25063-7_42
Ding, H., Wang, S., Xie, Z., Li, M., Ma, L.: A fine-grained vision and language representation framework with graph-based fashion semantic knowledge. Comput. Graphics (2023)
Esser, P., Rombach, R., Ommer, B.: Taming transformers for high-resolution image synthesis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12873–12883 (2021)
Gu, Y., et al.: VQFR: blind face restoration with vector-quantized dictionary and parallel decoder. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) ECCV 2022. LNCS, vol. 13678, pp. 126–143. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19797-0_8
Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Adv. Neural. Inf. Process. Syst. 33, 6840–6851 (2020)
Jung, M.: Correction: saturation-value based higher-order regularization for color image restoration. Multidimension. Syst. Signal Process. 34(2), 395–395 (2023)
Kouno, M., Nakae, K., Oba, S., Ishii, S.: Microscopic image restoration based on tensor factorization of rotated patches. Artif. Life Robot. 17, 417–425 (2013)
Li, H., Sheng, B., Li, P., Ali, R., Chen, C.P.: Globally and locally semantic colorization via exemplar-based broad-GAN. IEEE Trans. Image Process. 30, 8526–8539 (2021)
Li, X., Chen, C., Zhou, S., Lin, X., Zuo, W., Zhang, L.: Blind face restoration via deep multi-scale component dictionaries. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12354, pp. 399–415. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58545-7_23
Li, X., Zhang, S., Zhou, S., Zhang, L., Zuo, W.: Learning dual memory dictionaries for blind face restoration. IEEE Trans. Pattern Anal. Mach. Intell. 45(5), 5904–5917 (2023). https://doi.org/10.1109/TPAMI.2022.3215251
Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., Timofte, R.: SwinIR: image restoration using swin transformer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1833–1844 (2021)
Lugmayr, A., Danelljan, M., Romero, A., Yu, F., Timofte, R., Van Gool, L.: Repaint: inpainting using denoising diffusion probabilistic models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11461–11471 (2022)
Razavi, A., van den Oord, A., Vinyals, O.: Generating diverse high-fidelity images with VQ-VAE-2. In: Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8–14, 2019, Vancouver, BC, Canada, pp. 14837–14847 (2019)
Saharia, C., Ho, J., Chan, W., Salimans, T., Fleet, D.J., Norouzi, M.: Image super-resolution via iterative refinement. IEEE Trans. Pattern Anal. Mach. Intell. 45(4), 4713–4726 (2023). https://doi.org/10.1109/TPAMI.2022.3204461
Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., Ganguli, S.: Deep unsupervised learning using nonequilibrium thermodynamics. In: International Conference on Machine Learning, pp. 2256–2265. PMLR (2015)
Van Den Oord, A., Vinyals, O., et al.: Neural discrete representation learning. Advances in Neural Information Processing Systems, vol. 30 (2017)
Wang, X., Li, Y., Zhang, H., Shan, Y.: Towards real-world blind face restoration with generative facial prior. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9168–9178 (2021)
Wang, Z., Zhang, J., Chen, R., Wang, W., Luo, P.: Restoreformer: high-quality blind face restoration from undegraded key-value pairs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17512–17521 (2022)
Wen, Y., et al.: Structure-aware motion deblurring using multi-adversarial optimized cyclegan. IEEE Trans. Image Process. 30, 6142–6155 (2021)
Xie, Z., et al.: Boosting night-time scene parsing with learnable frequency. IEEE Trans. Image Process. 32, 2386–2398 (2023)
Xie, Z., Zhang, W., Sheng, B., Li, P., Chen, C.P.: BAGFN: broad attentive graph fusion network for high-order feature interactions. IEEE Trans. Neural Networks Learn. Syst. (2021)
Xing, W., Egiazarian, K.: Residual swin transformer channel attention network for image demosaicing. In: 2022 10th European Workshop on Visual Information Processing (EUVIP), pp. 1–6. IEEE (2022)
Yang, T., Ren, P., Xie, X., Zhang, L.: Gan prior embedded network for blind face restoration in the wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 672–681 (2021)
Yue, Z., Loy, C.C.: Difface: blind face restoration with diffused error contraction. arXiv preprint arXiv:2212.06512 (2022)
Zhou, S., Chan, K., Li, C., Loy, C.C.: Towards robust blind face restoration with codebook lookup transformer. Adv. Neural. Inf. Process. Syst. 35, 30599–30611 (2022)
Zhu, F., et al.: Blind face restoration via integrating face shape and generative priors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7662–7671 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sun, Y., Wang, S., Li, H., Xie, Z., Li, M., Ding, Y. (2024). Degradation-Aware Blind Face Restoration via High-Quality VQ Codebook. In: Sheng, B., Bi, L., Kim, J., Magnenat-Thalmann, N., Thalmann, D. (eds) Advances in Computer Graphics. CGI 2023. Lecture Notes in Computer Science, vol 14495. Springer, Cham. https://doi.org/10.1007/978-3-031-50069-5_26
Download citation
DOI: https://doi.org/10.1007/978-3-031-50069-5_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-50068-8
Online ISBN: 978-3-031-50069-5
eBook Packages: Computer ScienceComputer Science (R0)