skip to main content
10.1145/3468081.3471133acmotherconferencesArticle/Chapter ViewAbstractPublication PagesacitConference Proceedingsconference-collections
research-article

Image Inpainting using Clustered Planar Structure Guidance

Published: 22 October 2021 Publication History

Abstract

This paper presents an effective method using clustered planar structure guidance for image inpainting. Our method concerns restoring the unknown area by clustering structures of related planes. It is employed to obtain precisely similar structures in the surrounding area of missing regions. The approach of our work contains four essential steps: Planar Guidance, Clustering Structures, Feature Localization, and Patch Matching. According to perspective scenes, we first extract vanishing points (vp1, vp2, and vp3) using RAndom SAmple Consensus (RANSAC) algorithm as planar guidance. Then, we cluster the structure lines of each planar into more categories using the integration between K-means++ and Elbow method. Gaussian filter and Hadamard products blend among structure categories in the feature localization. This feature position propagates the surrounding structure information into unknown areas. For completing the unknown area, we employ PatchMatch [2] algorithm to match between the unknown and its surrounding patches. In experiments, our results perform well in various structures and perspective scenes that have foreshortened planes.

References

[1]
David Arthur and Sergei Vassilvitskii. 2006. k-means++: The advantages of careful seeding. Technical Report. Stanford.
[2]
Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B Goldman. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28, 3 (2009), 24.
[3]
Antonio Criminisi, Patrick Pérez, and Kentaro Toyama. 2004. Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on image processing 13, 9 (2004), 1200–1212.
[4]
Jia-Bin Huang, Sing Bing Kang, Narendra Ahuja, and Johannes Kopf. 2014. Image completion using planar structure guidance. ACM Transactions on graphics (TOG) 33, 4 (2014), 1–10.
[5]
Tapas Kanungo, David M Mount, Nathan S Netanyahu, Christine D Piatko, Ruth Silverman, and Angela Y Wu. 2002. An efficient k-means clustering algorithm: Analysis and implementation. IEEE transactions on pattern analysis and machine intelligence 24, 7(2002), 881–892.
[6]
Jiaying Liu, Shuai Yang, Yuming Fang, and Zongming Guo. 2018. Structure-guided image inpainting using homography transformation. IEEE Transactions on Multimedia 20, 12 (2018), 3252–3265.
[7]
Dhendra Marutho, Sunarna Hendra Handaka, Ekaprana Wijaya, and Others. 2018. The determination of cluster number at k-mean using elbow method and purity evaluation on headline news. In 2018 International Seminar on Application for Technology of Information and Communication. IEEE, 533–538.
[8]
Ivan Molodetskikh, Mikhail Erofeev, and Dmitry Vatolin. 2019. Perceptually motivated method for image inpainting comparison. arXiv preprint arXiv:1907.06296(2019).
[9]
Shohei Mori, Jan Herling, Wolfgang Broll, Norihiko Kawai, Hideo Saito, Dieter Schmalstieg, and Denis Kalkofen. 2018. 3D PixMix: Image Inpainting in 3D Environments. In 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, 1–2.
[10]
Muhammad Ali Qureshi, Mohamed Deriche, Azeddine Beghdadi, and Asjad Amin. 2017. A critical survey of state-of-the-art image inpainting quality assessment metrics. Journal of Visual Communication and Image Representation 49 (2017), 177–191.
[11]
Irawati Nurmala Sari, Emiko Horikawa, and Weiwei Du. 2021. Interactive Image Inpainting of Large-Scale Missing Region. IEEE Access 9(2021), 56430–56442.
[12]
Alexandru Telea. 2004. An image inpainting technique based on the fast marching method. Journal of graphics tools 9, 1 (2004), 23–34.
[13]
Philipp Tiefenbacher, Viktor Bogischef, Daniel Merget, and Gerhard Rigoll. 2015. Subjective and objective evaluation of image inpainting quality. In 2015 IEEE International Conference on Image Processing (ICIP). IEEE, 447–451.
[14]
Rafael Grompone Von Gioi, Jeremie Jakubowicz, Jean-Michel Morel, and Gregory Randall. 2008. LSD: A fast line segment detector with a false detection control. IEEE transactions on pattern analysis and machine intelligence 32, 4(2008), 722–732.
[15]
Lin Zhang, Lei Zhang, Xuanqin Mou, and David Zhang. 2011. FSIM: A feature similarity index for image quality assessment. IEEE transactions on Image Processing 20, 8 (2011), 2378–2386.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ACIT '21: Proceedings of the the 8th International Virtual Conference on Applied Computing & Information Technology
June 2021
147 pages
ISBN:9781450384933
DOI:10.1145/3468081
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 October 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Clustering Structures
  2. Feature Localization
  3. Image Inpainting
  4. Patch Matching
  5. Vanishing Points

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

ACIT 2021

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)5
  • Downloads (Last 6 weeks)1
Reflects downloads up to 10 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Weighted Similarity-Confidence Laplacian Synthesis for High-Resolution Art Painting CompletionApplied Sciences10.3390/app1406239714:6(2397)Online publication date: 12-Mar-2024
  • (2024)Handling Massive Sparse Data in Recommendation SystemsJournal of Information & Knowledge Management10.1142/S021964922450021723:03Online publication date: 23-Jan-2024
  • (2023)Structure-Texture Consistent Painting Completion for ArtworksIEEE Access10.1109/ACCESS.2023.325289211(27369-27381)Online publication date: 2023
  • (2022)Edge-enhanced GAN with Vanishing Points for Image Inpainting2022 23rd ACIS International Summer Virtual Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD-Summer)10.1109/SNPD-Summer57817.2022.00027(113-118)Online publication date: Jul-2022
  • (2022)Image Inpainting using Automatic Structure Propagation with Auxiliary Line Construction2022 23rd ACIS International Summer Virtual Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD-Summer)10.1109/SNPD-Summer57817.2022.00026(107-112)Online publication date: Jul-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media