Skip to main content
SpringerLink
Log in

Blind image inpainting quality assessment using local features continuity

  • 1197: Advances in Soft Computing Techniques for Visual Information-based Systems
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper deals with Blind Inpainted Image Quality Assessment BIIQA. Herein, we propose a new method that exploits the continuity of features around the boundaries of the retouched area. Indeed, we believe that the quality of inpainted images depends on how the edges and textures have been reproduced inside the hole. Besides, this concept has been formalized by the fact that features should be reproduced inside and outside the hole with respect to structures continuity. Furthermore, one could compare these features in terms of continuity and estimate the global quality of the inpainted image. And since the local structures are represented by patches, we proposed as a secondary contribution, an improvement of a patch classification algorithm. The strength of this metric unlike most existing IIQA metrics, is that it is completely blind and does not require any reference, making it well suited to the inpainting assessment, where reference is usually unavailable. The proposed BIIQA has been tested on TUM-IID database, where the results of four commonly used inpainting algorithms are provided and compared against IIQA state-of-the-art. The obtained results show clearly that our method outperforms the existing ones.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Ardis PA, Singhal A (2009) Visual salience metrics for image inpainting. In: Rabbani M, Stevenson RL (eds) Proceedings of SPIE 7257, Visual Communications and Image Processing, San Jose, CA. https://doi.org/10.1117/12.808942, p 72571W

  2. Bugeau A, Bertalmio M, Caselles V, Sapiro G (2010) A comprehensive framework for image inpainting. IEEE Trans Image Process 19(10):2634–2645

    Article  MathSciNet  Google Scholar 

  3. Dang TT, Beghdadi A, Larabi M-C (2013) Metric for visual coherence evaluation of color image restoration. In: 2013 colour and visual computing symposium (CVCS). https://doi.org/10.1109/CVCS.2013.6626268. IEEE, Norway, pp 1–6

  4. Getreuer P (2012) Total variation inpainting using split bregman. Image Processing On Line 2:147–157

    Article  Google Scholar 

  5. Gong H, Hang H (1994) Scene Analysis for DCT Image Coding, Signal processing of HDTV. Elsevier, Amsterdam, pp 425–434. https://doi.org/10.1016/B978-0-444-81844-7.50052-8

    Google Scholar 

  6. Guillemot C, Le Meur O (2014) Image Inpainting : Overview and Recent Advances. IEEE Signal Processing Magazine 31(1):127–144. https://doi.org/10.1109/MSP.2013.2273004

    Article  Google Scholar 

  7. Herling J, Broll W (2012) Pixmix: A real-time approach to high-quality diminished reality in International Symposium on Mixed and Augmented Reality. IEEE, Piscataway, pp 141–150

    Google Scholar 

  8. Hu W, Ye Y, Zeng F, Meng J (2018) A new method of thangka image inpainting quality assessment, J. Vis. Commun image r. https://doi.org/10.1016/j.jvcir.2018.12.045

  9. Isogawa M, Mikami D, Takahashi K, Kojima A (2016) Eye gaze analysis and learning-to-rank to obtain the most preferred result in image inpainting. In: 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ. https://doi.org/10.1109/ICIP.2016.7533018, pp 3538–3542

  10. Isogawa M, Mikami D, Takahashi K, et al. (2019) Image quality assessment for inpainted images via learning to rank. Multimed Tools Appl 78:1399–1418. https://doi.org/10.1007/s11042-018-6186-z

    Article  Google Scholar 

  11. Mahalingam VV, Cheung S-CS (2010) Eye tracking based perceptual image inpainting quality analysis, IEEE, Hong Kong. https://doi.org/10.1109/ICIP.2010.5653640

  12. Oncu AI, Deger F, Hardeberg JY (2012) Evaluation of digital inpainting quality in the context of artwork restoration. In: Proceeding of the 12th International Conference on Computer Vision. https://doi.org/10.1007/978-3-642-33863-2_58, pp 561–570

  13. Qureshi M, Deriche M, Beghdadi A, Amin A (2017,) A critical survey of state-of-the-art image inpainting quality assessment metrics. Journal of Visual Communication and Image Representation. 49. https://doi.org/10.1016/j.jvcir.2017.09.006

  14. Tiefenbacher P, Bogischef V, Merget D, Rigoll G (2015) Subjective and objective evaluation of image inpainting quality, IEEE, Canada

  15. Viacheslav V, Vladimir F, Vladimir M, Nikolay G, Roman S, Valentin F (2014) Low-level features for inpainting quality assessment. In: 2014 12th International Conference on Signal Processing (ICSP) Hangzhou. https://doi.org/10.1109/ICOSP.2014.7015082, pp 643–647

  16. Voronin VV, Vladimir A, Frantc VI, Marchuk AI, Sherstobitov K (2015) Egiazarian No-reference visual quality assessment for image inpainting SPIE, 9399 Image Processing: Algorithms and Systems XIII. https://doi.org/10.1117/12.2076507

  17. Wang S, Li H, Zhu X, Li P (2008) An evaluation index based on parameter weight for image inpainting quality, IEEE, Hunan. https://doi.org/10.1109/ICYCS.2008.461

  18. Xu Z, Sun J (2010) Image inpainting by patch propagation using patch sparsity. IEEE Trans Image Process 19(5):1153–1165

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laboratory of Image Processing and Radiation(LTIR), Faculty of Electronics and Computer Science, University of Science and Technology Houari Boumediene (USTHB), Algiers, Algeria

    Amine Mohamed Rezki & Amina Serir

  2. Laboratory of Information Processing and Transmission (L2TI), Institut Galilée, Université Sorbonne Paris Nord, 93430, Villetaneuse, France

    Azeddine Beghdadi

Authors
  1. Amine Mohamed Rezki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amina Serir
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Azeddine Beghdadi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amine Mohamed Rezki.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rezki, A.M., Serir, A. & Beghdadi, A. Blind image inpainting quality assessment using local features continuity. Multimed Tools Appl 81, 9225–9244 (2022). https://doi.org/10.1007/s11042-021-11872-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11872-2

Keywords

Search

Navigation