Skip to main content
SpringerLink
Log in

Image Inpainting Based on Structural Tensor Edge Intensity Model

  • Research Article
  • Published:
International Journal of Automation and Computing Aims and scope Submit manuscript

Abstract

In the exemplar-based image inpainting approach, there are usually two major problems: the unreasonable calculation of priority and only considering the color features in the patch lookup strategy. In this paper, we propose an image inpainting approach based on the structural tensor edge intensity model. First, we use the progressive scanning inpainting method to avoid the image filling order being affected by the priority function. Then, we use the edge intensity model to build the patches similarity function for correctly identifying the local image structure. Finally, the balance operator is used to restrict the excessive propagation of structural information to ensure the correct structural reconstruction. The experimental results show that the our approach is comparable and even superior to some state-of-the-art inpainting algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. V. K. Ha, J. C. Ren, X. Y. Xu, S. Zhao, G. Xie, V. Masero, A. Hussain. Deep learning based single image super-resolution: A survey. International Journal of Automation and Computing, vol. 16, no. 4, pp. 413–426, 2019. DOI: https://doi.org/10.1007/s11633-019-1183-x.

    Article  Google Scholar 

  2. M. Wang, R. C. Hong, X. T. Yuan, S. C. Yan, T. S. Chua. Movie2Comics: Towards a lively video content presentation. IEEE Transactions on Multimedia, vol. 14, no. 3, pp. 858–870, 2012. DOI: https://doi.org/10.1109/TMM.2012.2187181.

    Article  Google Scholar 

  3. J. B. Huang, S. B. Kang, N. Ahuja, J. Kopf. Temporally coherent completion of dynamic video. ACM Transactions on Graphics, vol. 35, no. 6, Article number 196, 2016. DOI: https://doi.org/10.1145/2980179.2982398.

  4. M. Bertalmio, G. Sapiro, V. Caselles, C. Ballester. Image inpainting. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, ACM, New York, USA, pp. 417–424, 2000. DOI: https://doi.org/10.1145/344779.344972.

    Google Scholar 

  5. T. F. Chan and J. H. Shen. Nontexture inpainting by curvature-driven diffusions. Journal of Visual Communication and Image Representation, vol. 12, no. 4, pp. 436–449, 2001. DOI: https://doi.org/10.1006/jvci.2001.0487.

    Article  Google Scholar 

  6. J. H. Shen and T. F. Chan. Mathematical models for local nontexture inpaintings. SIAM Journal on Applied Mathematics, vol. 62, no. 3, pp. 1019–1043, 2002. DOI: https://doi.org/10.1137/s0036139900368844.

    Article  MathSciNet  Google Scholar 

  7. K. S. Li, Y. S. Wei, Z. Yang, W. H. Wei. Image inpainting algorithm based on TV model and evolutionary algorithm. Soft Computing, vol. 20, no. 3, pp. 885–893, 2016. DOI: https://doi.org/10.1007/s00500-014-1547-7.

    Article  Google Scholar 

  8. X. H. Yang, B. L. Guo, Z. L. Xiao, W. Liang. Improved structure tensor for fine-grained texture inpainting. Signal Processing: Image Communication, vol. 73, pp. 84–95, 2019. DOI: https://doi.org/10.1016/j.image.2018.02.006.

    Google Scholar 

  9. P. Buyssens, M. Daisy, D. Tschumperlé, O. Lézoray. Exemplar-based inpainting: Technical review and new heuristics for better geometric reconstructions. IEEE Transactions on Image Processing, vol. 24, no. 6, pp. 1809–1824, 2015. DOI: https://doi.org/10.1109/TIP.2015.2411437.

    MathSciNet  MATH  Google Scholar 

  10. X. F. Li, J. Wang, H. M. Liu, Z. H. Wang. Image inpainting using feature precedence and patch matching. Journal of Computer-Aided Design & Computer Graphics, vol. 28, no. 7, pp. 1131–1137, 2016. DOI: https://doi.org/10.3969/j.issn.1003-9775.2016.07.012. (in Chinese)

    Google Scholar 

  11. J. Wang, K. Lu, D. R. Pan, N. He, B. K. Bao. Robust object removal with an exemplar-based image inpainting approach. Neurocomputing, vol. 123, pp. 150–155, 2014. DOI: https://doi.org/10.1016/j.neucom.2013.06.022.

    Article  Google Scholar 

  12. Y. Liu, C. J. Liu, H. L. Zou, S. S. Zhou, Q. Shen, T. T. Chen. A novel exemplar-based image inpainting algorithm. In Proceedings of International Conference on Intelligent Networking and Collaborative Systems, IEEE, Taipei, China, pp. 86–90, 2015. DOI: https://doi.org/10.1109/INCoS.2015.15.

    Google Scholar 

  13. X. H. Bi, H. M. Liu, G. M. Lu, J. Wei, Y. Chao. Exemplar-based image inpainting using automatic patch optimization. In Proceedings of the 2nd International Conference on Video and Image Processing, ACM, Hong Kong, China, pp. 128–133, 2018. DOI: https://doi.org/10.1145/3301506.3301549.

    Google Scholar 

  14. L. F. Zhang, L. C. Song, B. Du, Y. P. Zhang. Nonlocal low-rank tensor completion for visual data. IEEE Transactions on Cybernetics, published online. DOI: https://doi.org/10.1109/TCYB.2019.2910151.

  15. M. El Mallahi, A. Zouhri, A. El Affar, A. Tahiri, H. Qjidaa. Radial Hahn moment invariants for 2D and 3D image recognition. International Journal of Automation and Computing, vol. 15, no. 3, pp. 277–289, 2018. DOI: https://doi.org/10.1007/s11633-017-1071-1.

    Article  Google Scholar 

  16. T. Liu, Y. H. Wang, D. Z. Cheng. Dynamics and stability of potential hyper-networked evolutionary games. International Journal of Automation and Computing, vol. 14, no. 2, pp. 229–238, 2017. DOI: https://doi.org/10.1007/s11633-017-1056-0.

    Article  Google Scholar 

  17. A. Newson, A. Almansa, Y. Gousseau, P. Pérez. Non-local patch-based image inpainting. Image Processing on Line, vol. 7, pp. 373–385, 2017. DOI: https://doi.org/10.5201/ipol.2017.189.

    Article  MathSciNet  Google Scholar 

  18. C. Barnes, E. Shechtman, A. Finkelstein, D. B. Goldman. Patchmatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics, vol. 28, no. 3, Article number 24, 2009. DOI: https://doi.org/10.1145/1531326.1531330.

  19. H. M. Liu, X. H. Bi, G. M. Lu, W. L. Wang, Z. Y. Zhang. Screen window propagating for image inpainting. IEEE Access, vol. 6, pp. 61761–61772, 2018. DOI: https://doi.org/10.1109/ACCESS.2018.2876161.

    Article  Google Scholar 

  20. Criminisi A, Pérez P, Toyama K. Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on Image Processing, vol. 13, no. 9, pp. 1200–1212, 2004. DOI: https://doi.org/10.1109/TIP.2004.833105.

    Article  Google Scholar 

  21. Y. Xu, L. C. Yu, H. T. Xu, H. Zhang, T. Nguyen. Vector sparse representation of color image using quaternion matrix analysis. IEEE Transactions on Image Processing, vol. 24, no. 4, pp. 1315–1329, 2015. DOI: https://doi.org/10.1109/TIP.2015.2397314.

    Article  MathSciNet  Google Scholar 

  22. V. Naumova, K. Schnass. Dictionary learning from incomplete data for efficient image restoration. In Proceedings of the 25th European Signal Processing Conference, IEEE, Kos, Greece, pp. 1425–1429, 2017. DOI: https://doi.org/10.23919/EUSIPCO.2017.8081444.

    Google Scholar 

  23. Z. J. Yao, J. Bi, Y. X. Chen. Applying deep learning to individual and community health monitoring data: A survey. International Journal of Automation and Computing, vol. 15, no. 6, pp. 643–655, 2018. DOI: https://doi.org/10.1007/s11633-018-1136-9.

    Article  Google Scholar 

  24. N. Wang, S. H. Ma, J. Y. Li, Y. P. Zhang, L. F. Zhang. Multistage attention network for image inpainting. Pattern Recognition, vol. 106, Article number 107448, 2020. DOI: https://doi.org/10.1016/j.patcog.2020.107448.

Download references

Acknowlegements

This work was supported by National Science Foundation of China (Nos. 61401150, 61602157 and 61872311), Key Science and Technology Program of Henan Province (Nos.182102210053 and 202102210167), Excellent Young Teachers Program of Henan Polytechnic University (No. 2019XQG-02).

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, 454003, China

    Jing Wang, Yan-Hong Zhou, Hai-Feng Sima, Zhan-Qiang Huo & Ai-Zhong Mi

Authors
  1. Jing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan-Hong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hai-Feng Sima
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhan-Qiang Huo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ai-Zhong Mi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hai-Feng Sima.

Additional information

Colored figures are available in the online version at https://link.springer.com/journal/11633

Jing Wang received the B. Sc. degree in computer science and technology from Henan University of Science and Technology, China in 2006, and the Ph. D. degree in computer application technology from College of Computing and Communication Engineering, Graduate University of Chinese Academy of Science, China in 2012. Currently, she is an associate professor in College of Computer Science and Technology, Henan Polytechnic University, China.

Her research interests include image processing computer vision and machine learning.

Yan-Hong Zhou received the B. Eng. degree in computer science and technology from Fuyang Teachers College, China in 2018. Currently, she is a master student in software engineering at College of Computer Science and Technology, Henan Polytechnic University, China.

Her research interests include image processing and computer vision.

Hai-Feng Sima received the B. Eng. and M. Eng. degrees in computer science from Zhengzhou University, China in 2004 and 2007, respectively, and the Ph. D. degree in software and theory from Beijing Institute of Technology, China in 2015. Since 2007, he has been with Faculty of Henan Polytechnic University, China, and is currently a lecturer with College of Computer Science and Technology, Henan Polytechnic University, China.

His current research interests include pattern recognition, image processing, image segmentation and image classification.

Zhan-Qiang Huo received the B. Sc. degree in mathematics and applied mathematics from the Hebei Normal University of Science and Technology, China in 2003. He received the M. Sc. degree in computer software and theory and the Ph. D. degreD. degree in circuit and system from Yanshan University, China in 2006 and 2009. Currently, he is an associate professor in the College of Computer Science and Technology, Henan Polytechnic University, China. His research interests include computer vision and machine learning.

Ai-Zhong Mi received the M. Sc. degree in computer and application from Guangxi University, China in 2005, and the Ph. D. degree in computer application technology from University of Science and Technology Beijing, China in 2009. He is currently an associate professor in College of Computer Science and Technology, Henan Polytechnic University, China.

His research interests include pattern recognition and network security.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, J., Zhou, YH., Sima, HF. et al. Image Inpainting Based on Structural Tensor Edge Intensity Model. Int. J. Autom. Comput. 18, 256–265 (2021). https://doi.org/10.1007/s11633-020-1256-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11633-020-1256-x

Keywords

Search

Navigation