Skip to main content
SpringerLink
Log in

Rate-distortion optimized image compression based on image inpainting

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Inspired by recent advancements in image inpainting techniques, an image coding framework is proposed in this paper. In the framework, an original image is analyzed at the encoder side such that a number of the regions are skipped intentionally. A drop map is extracted and compressed into the generated bit stream to indicate the skipped regions. The image is recovered through the inpainting process by taking advantage of the available portion of the decoded image and the drop map at the decoder. Furthermore, the rate-distortion optimization is introduced to select the blocks to be removed for the better performance. Only the blocks containing certain visual features and satisfying rate-distortion criterion are dropped. A practical system is constructed to verify the effectiveness of the compression approach. Evaluations have been made in comparison with baseline JPEG and H.264/AVC. Compared to the baseline JPEG, the proposed algorithm obtains obvious visual quality improvements, as well as PSNR gains. Moreover, the proposed algorithm outperforms H.264/AVC intra coding under the low bit rates.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Bastani V, Helfroush MS, Kasiri K (2010) Image compression based on spatial redundancy removal and image inpainting. J Zhejiang Univ Sci C 11(2):92–100

    Article  Google Scholar 

  2. Bertalmio M, Sapiro G, Caselles V, Ballester C (2000) Image inpainting. In Proc. ACM SIGGRAPH, pp. 417–424

  3. Bertalmio M, Vese L, Sapiro G, Osher S (2003) Simultaneous structure and texture image inpainting. IEEE Trans Image Process 12(8):882–889

    Article  Google Scholar 

  4. Bertozzi, Esedoglu S, Gillette A (2007) Inpainting of binary images using the Cahn-Hilliard equation. IEEE Trans Image Process 16(1):285–291

    Article  MathSciNet  MATH  Google Scholar 

  5. Bugeau, Bertalmio M, Caselles V, Sapiro G (2010) A comprehensive framework for image inpainting. IEEE Trans Image Process 19(10):2634–2645

    Article  MathSciNet  Google Scholar 

  6. Cai JF, Chan RH, Shen Z (2008) A framelet-based image inpainting algorithm. Appl Comput Harmon Anal 24(2):131–149

    Article  MathSciNet  MATH  Google Scholar 

  7. Cai JF, Chan RH, Shen Z (2010) Simultaneous cartoon and texture inpainting. Inverse Probl Image 4(3):379–395

    Article  MathSciNet  MATH  Google Scholar 

  8. Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8(6):679–698

    Article  Google Scholar 

  9. Chan TF, Shen J, Zhou H-M (2006) Total variation wavelet inpainting. J Math Imaging Vision 25(1):107–125

    Article  MathSciNet  Google Scholar 

  10. Criminisi, Perez P, Toyama K (2004) Region filling and object removal by exemplar-based inpainting. IEEE Trans Image Process 13(9):1200–1212

    Article  Google Scholar 

  11. Dobrosotskaya JA, Bertozzi AL (2008) A wavelet-laplace variational technique for image deconvolution and inpainting. IEEE Trans Image Process 17(5):657–663

    Article  MathSciNet  Google Scholar 

  12. Elad M, Starck JL, Querre P, Donoho DL (2005) Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA). Appl Comput Harmon Anal 19(3):340–358

    Article  MathSciNet  MATH  Google Scholar 

  13. Fadili MJ, Starck JL, Murtagh F (2009) Inpainting and zooming using sparse representations. Comput J 52(1):64–79

    Article  Google Scholar 

  14. Grossauer H (2004) A combined PDE and texture synthesis approach to inpainting. In Proc. Eur. Conf. Comput. Vis. (ECCV’04), pp. 214–224

  15. Jiang W, Wang J, Yang J (2009) A novel algorithm of solving the optimal slope on rate-distortion curve for the given rate budget. J Donghua Univ 3:259–263

    Google Scholar 

  16. Komodakis N, Tziritas G (2007) Image completion using efficient belief propagation via priority scheduling and dynamic pruning. IEEE Trans Image Process 16(11):2649–2661

    Article  MathSciNet  Google Scholar 

  17. Li YR, Shen Lx, Suter BW. Adaptive inpainting algorithm based on DCT induced wavelet regularization. IEEE Trans Image Process 22(2):752-763

  18. Li YR, Shen L, Suter BW (2013) Adaptive inpainting algorithm based on DCT induced wavelet regularization. IEEE Trans Image Process 22(2):752–763

    Article  MathSciNet  Google Scholar 

  19. Liu D, Sun X, Wu F, Li S, Zhang Y-Q (2007) Image compression with edge-based inpainting. IEEE Trans Circ Syst Video Technol 17(10):1273–1287

    Article  Google Scholar 

  20. Liu D, Sun X, Wu F (2009) Edge-based inpainting and texture synthesis for image compression. Proc. Int. Conf. Multimedia & Expo pp. 1443–1447

  21. Masnou S (2002) Disocclusion: a variational approach using level lines. IEEE Trans Image Process 11(2):68–76

    Article  MathSciNet  Google Scholar 

  22. Ramchandran K, Vetterli M (1993) Best wavelet packet bases in a rate-distortion sense. IEEE Trans Image Process 2(2):160–175

    Article  Google Scholar 

  23. Rane SD, Sapiro G, Bertalmio M (2003) Structure and texture filling-in of missing image blocks in wireless transmission and compression applications. IEEE Trans Image Process 12(3):296–303

    Article  MathSciNet  Google Scholar 

  24. Reid MM, Millar RJ, Black ND (1997) Second-generation image coding: an overview. ACM Comput Surv 29(1):3–29

    Article  Google Scholar 

  25. Wang Z, Sheikh HR, Bovik AC (2002) No-reference perceptual quality assessment of JPEG compressed images. In Proc. IEEE Int. Conf. Image Processing (1):477–480

  26. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13:600–612

    Article  Google Scholar 

  27. Wang C, Sun X, Wu F, Xiong H (2006) Image compression with structure-aware inpainting. In Proc. IEEE Int. Symp. Circuits Syst. (ISCAS) pp. 1816–1819

  28. Wei Z, Ngan KN (2009) Spatio-temporal just noticeable distortion profile for grey scale image/video in DCT domain. IEEE Trans Circ Syst Video Technol 19(3):337–346

    Article  Google Scholar 

  29. Wen YW, Chan RH, Yip AM (2012) A primal–dual method for total-variation-based wavelet domain inpainting. IEEE Trans Image Process 21(1):106–114

    Article  MathSciNet  Google Scholar 

  30. Wexler Y, Shechtman E, Irani M (2007) Space-time completion of video. IEEE Trans Pattern Anal Mach Intell 29(3):463–476

    Article  Google Scholar 

  31. Wu Yd, Zhang Hy, Sun Y, Guo Hy (2009) Two image compression schemes based on image inpainting. CSO, Sanya, Hainan, China (1):816–820

  32. Xiong Z, Sun X, Wu F (2007) Image coding with parameter-assistant inpainting. Proc Int Conf Image Process 2:369–372

    Google Scholar 

  33. Xiong Z, Sun X, Wu F (2010) Block-based image compression with parameter-assistant inpainting. IEEE Trans Image Process 19(6):1651–1657

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China (NSFC, 61401269, 61371125, 61205081), the Natural Science Foundation of Shanghai (14ZR147400), Shanghai Technology Innovation Project (10110502200, 11510500900), Innovation Program of Shanghai Municipal Education Commission (12ZZ176, 13YZ105), Project of Science and Technology Commission of Shanghai Municipality (10PJ1404500), Leading Academic Discipline Project of Shanghai Municipal Education Commission (J51303)

Author information

Authors and Affiliations

  1. School of Computer and Information Engineering, Shanghai University of Electric Power, Shanghai, 200090, China

    Wei Jiang

  2. 702, No.35, Gaoqing road 2878, Shanghai, 200123, China

    Wei Jiang

Authors
  1. Wei Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Jiang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, W. Rate-distortion optimized image compression based on image inpainting. Multimed Tools Appl 75, 919–933 (2016). https://doi.org/10.1007/s11042-014-2332-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-2332-4

Keywords

Search

Navigation