Abstract
Image inpainting is the process of removing the unwanted objects from the image or it refers to the restoration of the original image. Despite the fact that there are various ways for image inpainting, these traditional approaches have some limitations in terms of data loss, which the proposed methodology should be able to address. This paper introduces a hybrid image inpainting method, termed DKH, which is the combination of deep learning, KNN, and biharmonic functions. Three phases make up the proposed DKH technique. The creation of the residual image, which takes place in the first phase, is accomplished using a Deep Convolutional Neural Network (Deep CNN) that was trained using the whale-monarch butterfly optimization algorithm. The second phase is the formation of patches and generation of the reconstructed image using the neighbour searching phenomenon named K-nearest neighbours (KNN), where the patch with the shortest distance is chosen during patch extraction using the Bhattacharya distance. In the third phase, the harmonic image is a reconstruction using biharmonal technique. Finally, using the Bayes probabilistic-based fusion method, the results of the three steps of image inpainting are combined. The performance of the image inpainting based on the proposed DKH is evaluated in terms of PSNR, SDME, SSIM, and accuracy. The developed image inpainting method achieves the PSNR of 35.63 dB, maximal SDME of 95.48 dB, the maximal SSIM of 0.960, and the maximal accuracy of 0.921 using Corel-10 k dataset.
Similar content being viewed by others
Data availability
The datasets analysed during the current study are available in “http://www.ci.gxnu.edu.cn/cbir/Dataset.aspx”.
Abbreviations
- KNN:
-
K-nearest neighbours
- Deep CNN:
-
Deep convolutional neural network
- PDE:
-
Partial differential equation
- MARR:
-
Momentum adaptive and rank revealing
- TSLRA:
-
Two-stage low-rank approximation
- MRF:
-
Markov random field
- TGV:
-
Total generalized variation
- MC:
-
Matrix completion
- FC:
-
Fully connected
- SDME:
-
Sustainable decision making exercise
- PSNR:
-
Peak signal-to-noise ratio
- SSIM:
-
Structural similarity index measure
- DWT:
-
Discrete wavelet transform
- Content-based-CVA:
-
Content-based-cognitive visual attention
- DNN:
-
Deep neural network
- Whale-MBO:
-
Whale-monarch butterfly optimization
- MBO:
-
Monarch butterfly optimization
- WOA:
-
Whale optimization algorithm
References
Alexander, R., Alexey, S., Vladimir, I., Alexandr, A.K.: Deep convolutional neural networks for breast cancer histology image analysis. In: International conference image analysis and recognition ICIAR, pp. 737–744. Springer, Cham (2018)
Brendt, W.: Inpainting by joint optimization of linear combinations of exemplars. IEEE Signal Process. Lett. 18(1), 75–78 (2011)
Chinmayee, H.R., Anupama, A., Bhagyashree, P., Patil, M.R.: Image Inpainting using exemplar based technique with improvised data term. In: 2018 International Conference on Computational Techniques, Electronics and Mechanical Systems (CTEMS), pp. 162–166 (2018)
Christine, G., Meur, O.L.: Image inpainting. IEEE Signal Process. Mag. 31(1), 127–144 (2014)
Corel-10k and GHIM-10k Datasets Taken (2018) http://www.ci.gxnu.edu.cn/cbir/Dataset.aspx. Accessed Aug 2018
Criminisi, A., Perez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)
Damelin, S.B., Hoang, N.S.: On surface completion and image inpainting by biharmonic functions: numerical aspects. Int. J. Math. Math. Sci. (2018). https://doi.org/10.1155/2018/3950312
Desai, M., Ganatra, A.: Survey on gap filling in satellite images and inpainting algorithm. Int. J. Comput. Theory Eng. 4(3), 341–345 (2012)
Erkan, U., Serdar, E., Dang, N.H.T.: An iterative image inpainting method based on similarity of pixels values. In: Proceedings of 6th International Conference on Electrical and Electronics Engineering, pp. 107–111 (2019)
Fawzi, A., Samulowitz, H., Turaga, D. and Frossard, P., Image inpainting through neural networks hallucinations. In: Image, Video, and Multidimensional Signal Processing, pp. 1–5 (2016)
Ghorai, M., Mandal, S., Chanda, B.: A group-based image inpainting using patch refinement in MRF framework. IEEE Trans. Image Process. 27(2), 556–567 (2018)
Guo, Q., Gao, S., Zhang, X., Yin, Y., Zhang, C.: Patch-based image inpainting via two-stage low rank approximation. IEEE Trans. vis. Comput. Graph. 24(6), 2023–2036 (2018)
Janani, V.D., Haribaabu, V.: Terminating the spamming links and privacy guaranteed search logs. In: proceedings of 2013 International Conference on Information Communication and Embedded Systems (ICICES), pp. 394–397 (2013)
Jiahui, Y., Zhe, L., Jimei, Y., Xiaohui, S., Xin, L., Thomas, S.H.: Generative image inpainting with contextual attention. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5505–5514 (2018)
Jiao, L., Hao, W., Haodi, W., Rongfang, B.: Multi-scale semantic image inpainting with residual learning and GAN. Neurocomputing 331, 199–212 (2019)
Jin, D., Bai, X.: Patch-sparsity-based image inpainting through facet deduced directional derivative. IEEE Trans. Circ. Syst. Video Technol. (2018). https://doi.org/10.1109/TCSVT.2018.2839351
Jixiang, C., Zhidan, L.: Markov Random Field-based image inpainting with direction structure distribution analysis for maintaining structure coherence. Signal Process. 154, 182–197 (2019)
Komal, S.M., Vaidya, M.B.: Image in painting techniques: a survey. IOSR J. Comput. Eng. (IOSRJCE) 5(4), 45–49 (2012)
Mahesh, M., Bhanodia, P.: Image inpainting techniques for removal of object. In: In International Conference on Information Communication and Embedded Systems, pp. 1–4 (2014)
Mirjalili, S., Lewis, A.: The whale optimization algorithm. Adv. Eng. Softw. 95, 51–67 (2016)
Newson, A., Almansa, A., Matthieu, F., Yann, G., Patrick, P.: Video inpainting of complex scenes. SIAM J. Imaging Sci. 7(4), 1993–2019 (2014)
Pal, G., Sharma, V.: Survey on different techniques in image inpainting. Int. J. Inf. Mov. 2(8), 119–127 (2017)
Pandya, N., Bhailal, L.: A survey on image inpainting techniques. Int. J. Curr. Eng. Technol. 2(2), 1–18 (2013)
Thanh, D.N.H., Surya Prasath, V.B., Sergey, D., Le Minh, H.: An adaptive image inpainting method based on euler’s elastica with adaptive parameters estimation and the discrete gradient method. Signal Process. 178, 107797 (2021)
Vahid, K.A., Farzin, Y.: Exemplar-based image inpainting using svd-based approximation matrix and multi-scale analysis. Multimedia Tools Appl. 76(5), 7213–7234 (2017)
Wali, S., Zhang, H., Chang, H., Wu, C.: A new adaptive boosting total generalized variation (TGV) technique for image denoising and inpainting. J. vis. Commun. Image Represent. 59, 39–51 (2019)
Wang, G.G., Deb, S., Zhao, X., Cui, Z.: A new monarch butterfly optimization with an improved crossover operator. Oper. Res. 18, 731–755 (2016)
Yang, S., Liu, J., Fang, Y., Guo, Z.: Structure-guided image inpainting using homography transformation. IEEE Trans. Multimedia 20(12), 3252–3265 (2018)
Yu, Y., Fangneng, Z., Shijian, L., Jianxiong, P., Feiying, M., Xuansong, X., Chunyan, M.: WaveFill: a wavelet-based generation network for image inpainting. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 14114–14123 (2021)
Zheng, J., Qin, M., Yu, H., Wang, W.: An efficient truncated nuclear norm constrained matrix completion for image inpainting. In: Computer Graphics, pp. 97–106 (2018)
Zongben, X., Sun, J.: Image inpainting by patch propagation using patch sparsity. IEEE Trans. Image Process. 19(5), 1153–1165 (2010)
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Hudagi, M.R., Soma, S. & Biradar, R.L. DKH: a hybridized approach for image inpainting using Bayes probabilistic-based image fusion. Int J Intell Robot Appl 7, 149–163 (2023). https://doi.org/10.1007/s41315-022-00267-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41315-022-00267-7