skip to main content
10.1145/3468081.3471129acmotherconferencesArticle/Chapter ViewAbstractPublication PagesacitConference Proceedingsconference-collections
research-article

Image Inpainting using Orthogonal Viewpoints and Structure Consistency in Manhattan World

Published: 22 October 2021 Publication History

Abstract

This paper proposes a fast and straightforward method for restoring a damaged hole in the Manhattan world, which has three orthogonality with one unknown camera to produce a perspective view that can foreshorten planes of similar structure. There are two essential guidance for image inpainting algorithms: Orthogonal Viewpoints and Structure Consistency. First, due to perspective view, we build orthogonal viewpoints from an image by determining the vanishing points of different planes. Second, we present structure consistency for reconstructing unknown regions of possibly foreshortened planes due to the displacement vector of repeated structures in perspective planes. For image inpainting, PatchMatch [1] algorithm handles matching structures based on two prior-primary guided methods. Our experiment results prove that our approach completes image inpainting in challenging scenes, such as perspective views, repetitive structures, and facades.

References

[1]
Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B Goldman. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28, 3 (2009), 24.
[2]
Dorin Comaniciu and Peter Meer. 2002. Mean shift: A robust approach toward feature space analysis. IEEE Transactions on pattern analysis and machine intelligence 24, 5(2002), 603–619.
[3]
Antonio Criminisi, Patrick Pérez, and Kentaro Toyama. 2004. Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on image processing 13, 9 (2004), 1200–1212.
[4]
Jia-Bin Huang, Sing Bing Kang, Narendra Ahuja, and Johannes Kopf. 2014. Image completion using planar structure guidance. ACM Transactions on graphics (TOG) 33, 4 (2014), 1–10.
[5]
Xiaohu Lu, Jian Yaoy, Haoang Li, Yahui Liu, and Xiaofeng Zhang. 2017. 2-line exhaustive searching for real-time vanishing point estimation in Manhattan world. In 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 345–353.
[6]
Ivan Molodetskikh, Mikhail Erofeev, and Dmitry Vatolin. 2019. Perceptually motivated method for image inpainting comparison. arXiv preprint arXiv:1907.06296(2019).
[7]
Muhammad Ali Qureshi, Mohamed Deriche, Azeddine Beghdadi, and Asjad Amin. 2017. A critical survey of state-of-the-art image inpainting quality assessment metrics. Journal of Visual Communication and Image Representation 49 (2017), 177–191.
[8]
Irawati Nurmala Sari, Emiko Horikawa, and Weiwei Du. 2021. Interactive Image Inpainting of Large-Scale Missing Region. IEEE Access 9(2021), 56430–56442.
[9]
Chanop Silpa-Anan and Richard Hartley. 2008. Optimised KD-trees for fast image descriptor matching. In 2008 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1–8.
[10]
Alexandru Telea. 2004. An image inpainting technique based on the fast marching method. Journal of graphics tools 9, 1 (2004), 23–34.
[11]
Philipp Tiefenbacher, Viktor Bogischef, Daniel Merget, and Gerhard Rigoll. 2015. Subjective and objective evaluation of image inpainting quality. In 2015 IEEE International Conference on Image Processing (ICIP). IEEE, 447–451.
[12]
Rafael Grompone Von Gioi, Jeremie Jakubowicz, Jean-Michel Morel, and Gregory Randall. 2008. LSD: A fast line segment detector with a false detection control. IEEE transactions on pattern analysis and machine intelligence 32, 4(2008), 722–732.
[13]
Lin Zhang, Lei Zhang, Xuanqin Mou, and David Zhang. 2011. FSIM: A feature similarity index for image quality assessment. IEEE transactions on Image Processing 20, 8 (2011), 2378–2386.

Cited By

View all
  • (2023)High-Resolution Art Painting Completion using Multi-Region Laplacian Fusion2023 Sixth International Symposium on Computer, Consumer and Control (IS3C)10.1109/IS3C57901.2023.00016(28-31)Online publication date: Jun-2023
  • (2023)Structure-Texture Consistent Painting Completion for ArtworksIEEE Access10.1109/ACCESS.2023.325289211(27369-27381)Online publication date: 2023
  • (2022)Edge-enhanced GAN with Vanishing Points for Image Inpainting2022 23rd ACIS International Summer Virtual Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD-Summer)10.1109/SNPD-Summer57817.2022.00027(113-118)Online publication date: Jul-2022
  • Show More Cited By

Index Terms

  1. Image Inpainting using Orthogonal Viewpoints and Structure Consistency in Manhattan World
        Index terms have been assigned to the content through auto-classification.

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Other conferences
        ACIT '21: Proceedings of the the 8th International Virtual Conference on Applied Computing & Information Technology
        June 2021
        147 pages
        ISBN:9781450384933
        DOI:10.1145/3468081
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 22 October 2021

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. Image Inpainting
        2. Orthogonal Viewpoints
        3. Structure Consistency
        4. Vanishing Points

        Qualifiers

        • Research-article
        • Research
        • Refereed limited

        Conference

        ACIT 2021

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)4
        • Downloads (Last 6 weeks)1
        Reflects downloads up to 10 Feb 2025

        Other Metrics

        Citations

        Cited By

        View all
        • (2023)High-Resolution Art Painting Completion using Multi-Region Laplacian Fusion2023 Sixth International Symposium on Computer, Consumer and Control (IS3C)10.1109/IS3C57901.2023.00016(28-31)Online publication date: Jun-2023
        • (2023)Structure-Texture Consistent Painting Completion for ArtworksIEEE Access10.1109/ACCESS.2023.325289211(27369-27381)Online publication date: 2023
        • (2022)Edge-enhanced GAN with Vanishing Points for Image Inpainting2022 23rd ACIS International Summer Virtual Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD-Summer)10.1109/SNPD-Summer57817.2022.00027(113-118)Online publication date: Jul-2022
        • (2022)Image Inpainting using Automatic Structure Propagation with Auxiliary Line Construction2022 23rd ACIS International Summer Virtual Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD-Summer)10.1109/SNPD-Summer57817.2022.00026(107-112)Online publication date: Jul-2022

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format.

        HTML Format

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media