Skip to main content
SpringerLink
Log in

Finger vein image inpainting using neighbor binary-wasserstein generative adversarial networks (NB-WGAN)

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Traditional inpainting methods obtain poor performance for finger vein images with blurred texture. In this paper, a finger vein image inpainting method using Neighbor Binary-Wasserstein Generative Adversarial Networks (NB-WGAN) is proposed. Firstly, the proposed algorithm uses texture loss, reconstruction loss, and adversarial loss to constrain the network, which protects the texture in the inpainting process. Secondly, the proposed NB-WGAN is designed with a coarse-to-precise generator network and a discriminator network composed of two Wasserstein Generative Adversarial Networks with Gradient Penalty (WGAN-GP). The cascade of a coarse generator network and a precise generator network based on Poisson fusion can obtain richer information and get natural boundary connection. The discriminator consists of a global WGAN-GP and a local WGAN-GP, which enforces consistency between the entire image and the repaired area. Thirdly, a training dataset is designed by analyzing the locations and sizes of the damaged finger vein images in practical applications (i.e., physical oil dirt, physical finger molting, etc). Experimental results show that the performance of the proposed algorithm is better than traditional inpainting methods including Curvature Driven Diffusions algorithm without texture constraints, a traditional inpainting algorithm with Gabor texture constraints, and a WGAN inpainting algorithm based on attention mechanism without texture constraints.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Liu T, Liu H, Chen ZZ et al (2018) Fast blind instrument function estimation method for industrial infrared spectrometers. IEEE Trans Industr Inform 14(12):5268–5277

    Google Scholar 

  2. Liu T, Liu H, Li YF et al (2019) Efficient blind signal reconstruction with wavelet transforms regularization for educational robot infrared vision sensing. IEEE/ASME Trans Mechatron 24(1):384–394

    Article  MathSciNet  Google Scholar 

  3. Liu T, Liu H, Li YF et al (2020) Flexible FTIR spectral imaging enhancement for industrial robot infrared vision sensing. IEEE Trans Industr Inform 16(1):544–554

    Article  Google Scholar 

  4. Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Physica D Nonlinear Phenomena 60(1-4):259–268

    Article  MathSciNet  Google Scholar 

  5. Chan TF, Shen JH (2001) Nontexture inpainting by curvature-driven diffusions. J Vis Commun Image Represent 12(4):436–449

    Article  Google Scholar 

  6. Criminisi A, Perez P, Toyama K (2004) Region filling and object removal by exemplar-based image inpainting. IEEE Trans Image Process 13(9):1200–1212

    Article  Google Scholar 

  7. Yang H, Shen L, Yao Y D, et al. (2020) Finger vein image inpainting with gabor texture constraints. IEEE Access 8:83041–83051

    Article  Google Scholar 

  8. Okajima K (1998) The gabor function extracts the maximum information from input local signals. Neural Netw 11(3):435–439

    Article  Google Scholar 

  9. Liu T, Li YF, Liu H et al (2019) RISIR: Rapid infrared spectral imaging restoration model for industrial material detection in intelligent video systems. IEEE Trans Industr Inform. https://doi.org/10.1109/TII.2019.2930463

  10. Fu Y, Xiang T, Jiang YG et al (2018) Recent advances in zero-shot recognition: toward data-efficient understanding of visual content. IEEE Signal Process Mag 35(1):112–125

    Article  Google Scholar 

  11. Ren S, He K, Girshick R et al (2017) Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal I Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  12. Salimans T, Goodfellow I, Zaremba W et al (2016) Improved techniques for training gans. Adv Neural Inf Process Syst 29:2234–2242

    Google Scholar 

  13. Iizuka S, Simo-Serra E, Ishikawa H (2017) Globally and locally consistent image completion. ACM Trans Graph 36(4):1–14

    Article  Google Scholar 

  14. Li Z, Liu H, Zhang Z et al (2021) Learning knowledge graph embedding with heterogeneous relation attention networks. In: IEEE Transactions on neural networks and learning systems, https://doi.org/10.1109/TNNLS.2021.3055147

  15. Wang Q, Fan H, Zhu L et al (2019) Deeply supervised face completion with multi-context generative adversarial network. IEEE Signal Proc Lett 26(3):400–404

    Article  Google Scholar 

  16. Gao R, Grauman K (2017) On-demand learning for deep image restoration. In: IEEE International conference on computer vision, pp 1095–1104

  17. Pathak D, Krahenbuhl P, Donahue J, et al. (2016) Context encoders: feature learning by inpainting. In: IEEE Conference on computer vision and pattern recognition, pp 2536–2544

  18. Yu J, Lin Z, Yang J, et al. (2018) Generative image inpainting with contextual attention. In: IEEE/CVF Conference on computer vision and pattern recognition, pp 5505–5514

  19. Gulrajani I, Ahmed F, Arjovsky M, et al. (2017) Improved Training of Wasserstein GANs. Adv Neural Inf Process Syst 30:5769–5779

    Google Scholar 

  20. Hore A, Ziou D (2010) Image quality metrics: PSNR vs SSIM. In: 20th International conference on pattern recognition, pp 2366–2369

  21. Zhang Z, Li Z, Liu H et al (2020) Multi-scale dynamic convolutional network for knowledge graph embedding. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2020.3005952

  22. Liu H, Fang S, Zhang Z et al (2021) MFDNet: Collaborative Poses Perception and Matrix Fisher Distribution for Head Pose Estimation. IEEE Trans Multimedia. https://doi.org/10.1109/TMM.2021.3081873

  23. Yu F, Koltun V (2016) Multi-scale context aggregation by dilated convolutions. ICLR. arXiv:1511.07122

  24. Clevert DA, Unterthiner T, Hochreiter S (2015) Fast and accurate deep network learning by exponential linear units (ELUs). Comput Sci. arXiv:1511.07289

  25. Takiyama R, Ono N (1989) A least square error estimation of the center and radii of concentric arcs. Pattern Recogn Lett 10(4):237–242

    Article  Google Scholar 

  26. Hamouchene I, Aouat S (2014) A cognitive approach for texture analysis using neighbors-based binary patterns. In: IEEE International conference on cognitive informatics and cognitive computing, pp 94–99

  27. Pollard D (1991) Asymptotics for least absolute deviation regression estimators. Econ Theory 7 (2):186–199

    Article  MathSciNet  Google Scholar 

  28. Rez P, Gangnet M, Blake A (2003) Poisson image editing. Acm Trans Graph 22(3):313–318

    Article  Google Scholar 

  29. Isola P, Zhu YJ, Zhou T et al (2017) Image-to-image translation with conditional adversarial networks. In: IEEE Conference on computer vision and pattern recognition, pp 1125–1134

  30. Lidong H, Wei Z, Jun W et al (2015) Combination of contrast limited adaptive histogram equalisation and discrete wavelet transform for image enhancement. Image Proc Iet 9(10):908–915

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Communication Engineering, Hangzhou Dianzi University, Hangzhou, 310018, China

    Hanqiong Jiang & Lei Shen

  2. College of Engineering, Architecture and Technology (CEAT), Oklahoma State University, Stillwater, OK, 74078, USA

    Huaxia Wang

  3. Stevens Institute of Technology, Hoboken, NJ, 07030, USA

    Yudong Yao

  4. Top Glory Tech Limited Company, Hangzhou, 310000, China

    Guodong Zhao

Authors
  1. Hanqiong Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lei Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huaxia Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yudong Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guodong Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lei Shen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, H., Shen, L., Wang, H. et al. Finger vein image inpainting using neighbor binary-wasserstein generative adversarial networks (NB-WGAN). Appl Intell 52, 9996–10007 (2022). https://doi.org/10.1007/s10489-021-03017-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-03017-7

Keywords

Search

Navigation