Skip to main content
SpringerLink
Log in

A two-stage enhancement network with optimized effective receptive field for speckle image reconstruction

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Reconstructing target objects from strong speckle images is a key step for solving complex inverse scattering imaging problems. Deep learning (DL) methods are very effective for producing high quality object reconstruction, especially for speckle image reconstruction (SIR). Understanding the relationship between DL network structures and reconstruction results helps improve the reconstruction quality. Although previous studies have explored this issue, few of them considered dilated convolution adjustment and effective receptive field optimization of DL networks in image reconstruction for improving the reconstruction quality. In this paper, we propose a two stage enhancement network for speckle image reconstruction, in addition, we present an effective receptive field optimization method for maximizing the usage of the network capability. Specifically, in the first stage, we propose a growth model exploiting the dilation rates under the assumption that the central area pixels of images have a much bigger impact on the output field than the outer area pixels, and accordingly optimize the effective receptive field of the networks. Then, based on our growth model, in the second stage, the enhancement network jointly utilizes complementary information from the objective loss and perceptual loss when reconstructing objects. Extensive experiments show that our new network outperforms five state-of-the-art methods in the MAE, MSE, PSNR, and SSIM evaluating measures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Algorithm 1
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

  1. Anoop B, Girish G, Sudeep P, Rajan J (2019) Despeckling algorithms for optical coherence tomography images: a review, Advanced Classification Techniques for Healthcare Analysis :286–310

  2. Candes EJ, Li X, Soltanolkotabi M (2015) Phase retrieval via wirtinger flow: theory and algorithms. IEEE Trans Inf Theory 61(4):1985–2007

    Article  MathSciNet  MATH  Google Scholar 

  3. Chen J, Ying H, Liu X, Gu J, Feng R, Chen T, Gao H, Jian W (2020) A transfer learning based super-resolution microscopy for biopsy slice images: the joint methods perspective. IEEE/ACM Trans Comput Biol Bioinforma 18 (1):103–113

    Google Scholar 

  4. Chen T, Liu X, Feng R, Wang W, Yuan C, Lu W, He H, Gao H, Ying H, Chen DZ, Wu J (2021) Discriminative cervical lesion detection in colposcopic images with global class activation and local bin excitation. IEEE J Biomed Health Inf 26(4):1411–1421

    Article  Google Scholar 

  5. Deng X, Yang R, Xu M, Dragotti PL (2019) Wavelet domain style transfer for an effective perception-distortion tradeoff in single image super-resolution. In: ICCV, pp 3076–3085

  6. Gao H, Xu K, Cao M, Xiao J, Xu Q, Yin Y (2021) The deep features and attention mechanism-based method to dish healthcare under social IoT systems: an empirical study with a hand-deep local–global net. IEEE Trans Comput Soc Syst 9(1):336–347

    Article  Google Scholar 

  7. Gupta RK, Bruce GD, Powis SJ, Dholakia K (2020) Deep learning enabled laser speckle wavemeter with a high dynamic range. Laser Photonics Rev 14:2000120

    Article  Google Scholar 

  8. Horé A., Ziou D (2020) Image quality metrics: PSNR vs. SSIM

  9. Huang GB, Ramesh M, Berg T, Learned-Miller E (2008) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. In: Dans Workshop on Faces in ‘Real-Life’ Images: Detection, Alignment, and Recognition. https://hal.inria.fr/inria-00321923

  10. Hyun D, Brickson LL, Looby KT, Dahl JJ (2019) Beamforming and speckle reduction using neural networks. IEEE Trans Ultrason Ferroelectrics Freq Control 66(5):898–910

    Article  Google Scholar 

  11. Kakkava E, Rahmani B, Borhani N, Tegin U, Loterie D, Konstantinou G, Moser C, Psaltis D (2019) Imaging through multimode fibers using deep learning: the effects of intensity versus holographic recording of the speckle pattern. Opt Fiber Technol 52:101985

    Article  Google Scholar 

  12. Katz O, Heidmann P, Fink M, Gigan S (2014) Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations. Nat Photonics 8:784–790

    Article  Google Scholar 

  13. Kim M, Choi W, Choi Y, Yoon C, Choi W (2015) Transmission matrix of a scattering medium and its applications in biophotonics. Opt Express 23(10):12468–12668

    Article  Google Scholar 

  14. Lan T, Li K (2021) Efficient reconstruction of industrial images using optimized HMK splines. IEEE Trans Ind Inform 17(7):4657–4668

    Article  MathSciNet  Google Scholar 

  15. LeCun Y, Cortes C, Burges CJ (2010) MNIST handwritten digit database. AT&T Labs. http://yann.lecun.com/exdb/mnist. Accessed 1998

  16. Liu Y, Yu J, Han Y (2018) Understanding the effective receptive field in semantic image segmentation. Multimed Tools Appl 77:22159–22171

    Article  Google Scholar 

  17. Li Y, Xue Y, Tian L (2018) Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media. Optica 5(10):1181–1190

    Article  Google Scholar 

  18. Li S, Deng M, Lee J, Sinha A, Barbastathis G (2018) Imaging through glass diffusers using densely connected convolutional networks. Optica 5 (7):803–813

    Article  Google Scholar 

  19. Li Y, Cheng S, Xue Y, Tian L (2021) Displacement-agnostic coherent imaging through scatter with an interpretable deep neural network. Opt Express 29(2):2244–2257

    Article  Google Scholar 

  20. Luo W, Li Y, Urtasun R, Zemel R (2016) Understanding the effective receptive field in deep convolutional neural networks 30Th conference on neural information processing systems (NIPS)

  21. Mirza M, Osindero S (2014) Conditional generative adversarial nets. CoRR. arXiv:1411.1784

  22. Mishra S, Chen DZ, Hu XS (2020) A data-aware deep supervised method for retinal vessel segmentation. In: IEEE 17Th international symposium on biomedical imaging (ISBI), pp 1254–1257

  23. Mohan E, Rajeshi A, Sunitha G, Konduru RM, Auanija J, Babu LG (2021) A deep neural network learning-based speckle noise removal technique for enhancing the quality of synthetic-aperture radar images. Concurrency Computation Practice Experience., to be published

  24. Mosk AP, Lagendijk A, Lerosey G, Fink M (2012) Controlling waves in space and time for imaging and focusing in complex media. Nat Photonics 6(5):283–292

    Article  Google Scholar 

  25. Pathak D, Krahenbuhl P, Donahue J, Darrell T, Efros AA (2016) Context encoders: feature learning by inpainting. IEEE Conf Comput Vis Pattern Recognit :2536–2544

  26. Romera E, Alvarez JM, Bergasa LM, Arroyo R (2018) ERFNEt: Efficient residual factorized ConvNet for real-time semantic segmentation. IEEE Trans Intell Transp Syst 19(1):263–272

    Article  Google Scholar 

  27. Sanghvi Y, Kalepu Y, Khankhoje UK (2020) Embedding deep learning in inverse scattering problems. IEEE Trans Comput Imaging 6:46–56

    Article  Google Scholar 

  28. Santos MS, Kalantari NK (2020) Single image HDR reconstruction using a CNN with masked features and perceptual loss. ACM Trans Graph 39 (4):80:1–80:10

    Article  Google Scholar 

  29. Satat G, Tancik M, Raskar R (2020) Lensless imaging with compressive ultrafast sensing. IEEE Trans Comput Imaging 3(3):398–407

    Article  MathSciNet  Google Scholar 

  30. Sharma MK, Metzler CA, Nagesh S, Baraniuk RG, Cossairt O, Veeraraghavan A (2020) Inverse scattering via transmission matrices: broadband illumination and fast phase retrieval algorithms. IEEE Trans Comput Imaging 6:95–108

    Article  Google Scholar 

  31. Shensa MJ (1992) The discrete wavelet transform: Wedding the a trous and Mallat algorithms. IEEE Trans Signal Process 40(10):2464–2482

    Article  MATH  Google Scholar 

  32. Sinha A, Lee J, Li S, Barbastathis G (2017) Lensless computational imaging through deep learning. Optica 4(9):1117–1125

    Article  Google Scholar 

  33. Sun Y, Shi J, Sun L, Fan J, Zeng G (2019) Image reconstruction through dynamic scattering media based on deep learning. Opt Express 27(11):16032–16044

    Article  Google Scholar 

  34. Tan HL, Li Z, Tan YH, Rahardja S, Yeo C (2013) A perceptually relevant MSE-based image quality metric. IEEE Trans Image Process 22 (11):4447–4458

    Article  MathSciNet  MATH  Google Scholar 

  35. Uelwer T, Oberstra A, Harmeling S (2021) Phase retrieval using conditional generative adversarial networks. In: International conference on pattern recognition (ICPR), pp 10–15

  36. Vellekoop IM, Mosk AP (2007) Focusing coherent light through opaque strongly scattering media. Opt Lett 32(16):2309–2311

    Article  Google Scholar 

  37. Wang Z, Ji S (2018) Smoothed dilated convolutions for improved dense prediction. In: KDD, London, United Kingdom, pp 2486–2495

  38. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  39. Xiao H, Rasul K, Vollgraf R Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv:1708.07747

  40. Xiao J, Xu H, Gao H, Bian M, Li Y (2021) A weakly supervised semantic segmentation network by aggregating seed cues: the multi-object proposal generation perspective. ACM Trans Multimed Comput Commun Appl 17(1s):1–19

    Article  Google Scholar 

  41. Xu K, Ba JL, Kiros R, Courville A, Salakhutdinov R, Zemel R, Bengio Y (2015) Show, attend and tell: neural image caption generation with visual attention. In: Proceedings of machine learning research (PMLR), vol. 37, pp. 2048–2057

  42. Yang Y, Deng L, Jiao P, Chua Y, Pei J, Ma C, Li G (2020) Transfer learning in general lensless Imaging through scattering media. In: IEEE conference on industrial electronics and applications (ICIEA), pp 1132–1141

  43. Yao HM, Sha WEI, Jiang L (2019) Two-step enhanced deep learning approach for electromagnetic inverse scattering problems. IEEE Antennas Wirel Propag Lett 18(11):2254–2258

    Article  Google Scholar 

  44. Yao HM, Jiang L, Sha WEI (2020) Enhanced deep learning approach based on the deep convolutional encoder-decoder architecture for electromagnetic inverse scattering problems. IEEE Antennas Wirel Propag Lett 19(7):1211–1215

    Article  Google Scholar 

  45. Yoon S, Kim M, Jang M, Choi Y, Choi W, Kang S, Choi W (2020) Deep optical imaging within complex scattering media. Nat Rev 2:141–158

    Google Scholar 

  46. Yu F, Koltun V (2015) Multi-scale context aggregation by dilated convolutions. arXiv:1511.07122

  47. Yuan Z, Wang H Multiple scattering media imaging via end-to-end neural network. arXiv:1806.09968

  48. Zhou Z, Wang Y, Guo Y, Qi Y, Yu J (2020) Image quality improvement of hand-held ultrasound devices with a two-stage generative adversarial network. IEEE Trans Biomed Eng 67(1):298–311

    Article  Google Scholar 

  49. Zhu H, Cao Z, Lian L, Ye G, Gao H, Jian W (2022) Cariesnet: a deep learning approach for segmentation of multi-stage caries lesion from oral panoramic X-ray image. Neural Comput Applic 7:1–9

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grants (62031018, 61971227).

Author information

Authors and Affiliations

  1. Jiangsu Key Laboratory of Spectral Imaging and Intelligent Sense, Nanjing University of Science and Technology, Nanjing, 210094, Jiangsu, China

    Linli Xu, Jing Han & Lianfa Bai

  2. Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, 46556, IN, USA

    Peixian Liang & Danny Z. Chen

Authors
  1. Linli Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peixian Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jing Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lianfa Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Danny Z. Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jing Han or Lianfa Bai.

Ethics declarations

Conflict of Interests

Authors Linli Xu, Peixian Liang, Jing Han, Lianfa Bai and Danny Z. Chen declare that they have no conflicts of interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, L., Liang, P., Han, J. et al. A two-stage enhancement network with optimized effective receptive field for speckle image reconstruction. Multimed Tools Appl 82, 19923–19943 (2023). https://doi.org/10.1007/s11042-022-14208-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14208-w

Keywords

Search

Navigation