skip to main content
10.1145/3577530.3577544acmotherconferencesArticle/Chapter ViewAbstractPublication PagescsaiConference Proceedingsconference-collections
research-article

Thangka Image Inpainting Algorithm Based on Edge Constraint

Published: 30 March 2023 Publication History

Abstract

Thangka images have the characteristics of rich content, complex textures and bright colors. The existing deep learning inpainting algorithms have some problems in inpainting incomplete Thangka images, such as blurred boundary, distorted structure and unclear texture. To solve the above problems, this paper proposes a two-stage Thangka image inpainting model based on edge constraint, which consists of edge prediction model and texture generation model. First, the edge of incomplete Thangka is reconstructed by the generative adversarial network based on U-Net structure. In order to increase the ability of feature extraction, a hierarchical residual structure is constructed in U-NET. Second, with the constraint of filling edge, a texture generation model combining gated convolution and multi-scale attention is used to reconstruct Thangka texture. Here, multi-scale attention can effectively capture multiscale features to generate images consistent with the texture details of the ground truth images. Experiments demonstrate that the proposed model can achieve better results, and its subjective and objective evaluation indicators are superior than the current image inpainting algorithms.

References

[1]
Criminisi, A., Pérez, P., & Toyama, K. 2004. Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on image processing, 13(9), 1200-1212.
[2]
Barnes, C., Shechtman, E., Finkelstein, A., & Goldman, D. B. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph., 28(3), 24.
[3]
Hays, J., & Efros, A. A. 2007. Scene completion using millions of photographs. ACM Transactions on Graphics (ToG), 26(3), 4-es.
[4]
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. 2016. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2536-2544).
[5]
Iizuka, S., Simo-Serra, E., & Ishikawa, H. 2017. Globally and locally consistent image completion. ACM Transactions on Graphics (ToG), 36(4), 1-14.
[6]
Liu, G., Reda, F. A., Shih, K. J., Wang, T. C., Tao, A., & Catanzaro, B. 2018. Image inpainting for irregular holes using partial convolutions. In Proceedings of the European conference on computer vision (ECCV) (pp. 85-100).
[7]
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. S. 2019. Free-form image inpainting with gated convolution. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 4471-4480).
[8]
Nazeri, K., Ng, E., Joseph, T., Qureshi, F. Z., & Ebrahimi, M. 2019. Edgeconnect: Generative image inpainting with adversarial edge learning. arXiv preprint arXiv:1901.00212.
[9]
Ren, Y., Yu, X., Zhang, R., Li, T. H., Liu, S., & Li, G. 2019. Structureflow: Image inpainting via structure-aware appearance flow. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 181-190).
[10]
Ronneberger, O., Fischer, P., & Brox, T. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention (pp. 234-241). Springer, Cham.
[11]
Gao, S. H., Cheng, M. M., Zhao, K., Zhang, X. Y., Yang, M. H., & Torr, P. 2019. Res2net: A new multi-scale backbone architecture. IEEE transactions on pattern analysis and machine intelligence, 43(2), 652-662.
[12]
Hu, J., Shen, L., & Sun, G. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7132-7141).
[13]
Qin, J., Bai, H., & Zhao, Y. 2021. Multi-scale attention network for image inpainting. Computer Vision and Image Understanding, 204, 103155.
[14]
Jianchu She and Ying Liu. Facial Image Inpainting Algorithm Based on Attention Mechanism and Dual Discriminators. Journal of Image and Graphics, V ol. 10, No. 1, pp. 43-49, March 2022.

Cited By

View all

Index Terms

  1. Thangka Image Inpainting Algorithm Based on Edge Constraint
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Information & Contributors

          Information

          Published In

          cover image ACM Other conferences
          CSAI '22: Proceedings of the 2022 6th International Conference on Computer Science and Artificial Intelligence
          December 2022
          341 pages
          ISBN:9781450397773
          DOI:10.1145/3577530
          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 30 March 2023

          Permissions

          Request permissions for this article.

          Check for updates

          Author Tags

          1. Gated convolution
          2. Generative Adversarial Network
          3. Hierarchical residual network
          4. Multi-scale attention
          5. Thangka image inpainting

          Qualifiers

          • Research-article
          • Research
          • Refereed limited

          Conference

          CSAI 2022

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • 0
            Total Citations
          • 34
            Total Downloads
          • Downloads (Last 12 months)8
          • Downloads (Last 6 weeks)2
          Reflects downloads up to 27 Feb 2025

          Other Metrics

          Citations

          Cited By

          View all

          View Options

          Login options

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format.

          HTML Format

          Figures

          Tables

          Media

          Share

          Share

          Share this Publication link

          Share on social media