Skip to main content
SpringerLink
Log in
Book cover

International Joint Conference on Computer Vision, Imaging and Computer Graphics

VISIGRAPP 2020: Computer Vision, Imaging and Computer Graphics Theory and Applications pp 477–494Cite as

CNN-Based Deblurring of THz Time-Domain Images

  • Conference paper
  • First Online:

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1474))

Abstract

In recent years, terahertz (THz) time-domain imaging attracted significant attention and become a useful tool in many applications. A THz time-domain imaging system measures amplitude changes of the THz radiation across a range of frequencies so the absorption coefficient of the materials in the sample can be obtained. THz time-domain images represent 3D hyperspectral cubes with several hundred bands corresponding to different wavelengths i.e., frequencies. Moreover, a THz beam has a non-zero beam waist and therefore introduces band-dependent blurring effects in the resulting images accompanied by system-dependent noise. Removal of blurring effects and noise from the whole 3D hyperspectral cube is addressed in the current work. We will start by introducing THz beam shape effects and its formulation as a deblurring problem, followed by presenting a convolutional neural network (CNN)-based approach which is able to tackle all bands jointly. To the best of our knowledge, this is the first time that a CNN is used to remove the THz beam shape effects from all bands jointly of THz time-domain images. Experiments on synthetic images show that the proposed approach significantly outperforms conventional model-based deblurring methods and band-by-band approaches.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://mxnet.apache.org/.

References

  1. Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: A deep convolutional encoder-decoder architecture for image segmentation. arXiv preprint arXiv:1511.00561 (2015)

  2. Baxter, J.B., Guglietta, G.W.: Terahertz spectroscopy. Anal. Chem. 83(12), 4342–4368 (2011). https://doi.org/10.1021/ac200907z

    Article  Google Scholar 

  3. Bazrafkan, S., Thavalengal, S., Corcoran, P.: An end to end deep neural network for iris segmentation in unconstrained scenarios. Neural Netw. 106, 79–95 (2018). https://doi.org/10.1016/j.neunet.2018.06.011, http://www.sciencedirect.com/science/article/pii/S089360801830193X

  4. Bazrafkan, S., Van Nieuwenhove, V., Soons, J., De Beenhouwer, J., Sijbers, J.: Deep neural network assisted iterative reconstruction method for low dose ct. arXiv preprint arXiv:1906.00650 (2019)

  5. Chan, W.L., Deibel, J., Mittleman, D.M.: Imaging with terahertz radiation. Rep. Prog. Phys. 70(8), 1325–1379 (2007). https://doi.org/10.1088/0034-4885/70/8/r02

    Article  Google Scholar 

  6. Chen, T., et al.: Mxnet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems. arXiv preprint arXiv:1512.01274 (2015)

  7. Cosentino, A.: Terahertz and cultural heritage science: examination of art and archaeology. Technologies 4(1) (2016). https://doi.org/10.3390/technologies4010006, https://www.mdpi.com/2227-7080/4/1/6

  8. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007). https://doi.org/10.1109/TIP.2007.901238

    Article  MathSciNet  Google Scholar 

  9. Danielyan, A., Katkovnik, V., Egiazarian, K.: BM3D frames and variational image deblurring. IEEE Trans. Image Process. 21(4), 1715–1728 (2012). https://doi.org/10.1109/TIP.2011.2176954

    Article  MathSciNet  MATH  Google Scholar 

  10. Dhawan, A., Rangayyan, R., Gordon, R.: Image restoration by wiener deconvolution in limited-view computed tomography. Appl. Opt. 24, 4013 (1986). https://doi.org/10.1364/AO.24.004013

    Article  Google Scholar 

  11. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016). https://doi.org/10.1109/TPAMI.2015.2439281

    Article  Google Scholar 

  12. Duvillaret, L., Garet, F., Coutaz, J.L.: Influence of noise on the characterization of materials by terahertz time-domain spectroscopy. JOSA B 17, 452–461 (2000). https://doi.org/10.1364/JOSAB.17.000452

    Article  Google Scholar 

  13. Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the International Conference on Artificial Intelligence and Statistics (2010)

    Google Scholar 

  14. Guillet, J.P., et al.: Review of terahertz tomography techniques. J. Infrared Millim. Terahertz Waves 35(4), 382–411 (2014). https://doi.org/10.1007/s10762-014-0057-0, https://hal.archives-ouvertes.fr/hal-00968839

  15. Haddad, J.E., Bousquet, B., Canioni, L., Mounaix, P.: Review in terahertz spectral analysis. Trends Anal. Chem. 44, 98–105 (2013). https://doi.org/10.1016/j.trac.2012.11.009, http://www.sciencedirect.com/science/article/pii/S0165993613000022

  16. Hu, B.B., Nuss, M.C.: Imaging with terahertz waves. Opt. Lett. 20(16), 1716–1718 (1995). https://doi.org/10.1364/OL.20.001716, http://ol.osa.org/abstract.cfm?URI=ol-20-16-1716

  17. Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)

  18. Kato, N., et al.: The deep learning vision for heterogeneous network traffic control: proposal, challenges, and future perspective. IEEE Wirel. Commun. 24(3), 146–153 (2016)

    Article  Google Scholar 

  19. Kawase, K., Shibuya, T., Hayashi, S., Suizu, K.: THz Imaging techniques for nondestructive inspections. Comptes Rendus Physique 11(7), 510–518 (2010). https://doi.org/10.1016/j.crhy.2010.04.003, http://www.sciencedirect.com/science/article/pii/S1631070510000423, terahertz electronic and optoelectronic components and systems

  20. Kemp, M.C., Taday, P.F., Cole, B.E., Cluff, J.A., Fitzgerald, A.J., Tribe, W.R.: Security applications of terahertz technology. In: Terahertz for Military and Security Applications, vol. 5070, pp. 44–52 (2003). https://doi.org/10.1117/12.500491, https://doi.org/10.1117/12.500491

  21. Kingma, D.P., Ba, J.: Adam: A Method for Stochastic Optimization. arXiv preprint arXiv:1412.6980 (2014)

  22. Krishnan, D., Fergus, R.: Fast image deconvolution using hyper-laplacian priors. Adv. Neural Inf. Process. Syst. 1033–1041 (2009). http://papers.nips.cc/paper/3707-fast-image-deconvolution-using-hyper-laplacian-priors

  23. LeCun, Y., Bengio, Y., Hinton, G.E.: Deep learning. Nature 521(7553), 436–444 (2015). https://doi.org/10.1038/nature14539, https://doi.org/10.1038/nature14539

  24. Lemley, J., Bazrafkan, S., Corcoran, P.: Deep learning for consumer devices and services: pushing the limits for machine learning, artificial intelligence, and computer vision. IEEE Consum. Electron. Mag. 6(2), 48–56 (2017)

    Article  Google Scholar 

  25. Litjens, G., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)

    Article  Google Scholar 

  26. Ljubenović., M., Bazrafkan., S., Beenhouwer., J.D., Sijbers., J.: CNN-based deblurring of terahertz images. In: Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, pp. 323–330. INSTICC, SciTePress (2020). https://doi.org/10.5220/0008973103230330

  27. Lucy, L.B.: An iterative technique for the rectification of observed distributions. Astron. J. 79(6), 745–754 (1974)

    Article  Google Scholar 

  28. Mallat, S.: A Wavelet Tour of Signal Processing, Third Edition: The Sparse Way, 3rd edn. Academic Press Inc., Orlando, FL, USA (2008)

    Google Scholar 

  29. Mukherjee, S., Federici, J., Lopes, P., Cabral, M.: Elimination of Fresnel reflection boundary effects and beam steering in pulsed terahertz computed tomography. J. Infrared Millim. Terahertz Waves 34(9), 539–555 (2013). https://doi.org/10.1007/s10762-013-9985-3

    Article  Google Scholar 

  30. Pelt, D., Batenburg, K., Sethian, J.: Improving tomographic reconstruction from limited data using mixed-scale dense convolutional neural networks. J. Imaging 4(11), 128 (2018)

    Article  Google Scholar 

  31. Pickwell-MacPherson, E., Wallace, V.P.: Terahertz pulsed imaging - a potential medical imaging modality? Photodiagn. Photodyn. Ther. 6, 128–134 (2009)

    Article  Google Scholar 

  32. Popescu, D.C., Hellicar, A.D.: Point spread function estimation for a terahertz imaging system. EURASIP J. Adv. Signal Process. 2010(1), 575817 (2010). https://doi.org/10.1155/2010/575817, https://doi.org/10.1155/2010/575817

  33. Recur, B., et al.: Propagation beam consideration for 3D THz computed omography. Opt. Express 20(6), 5817–5829 (2012). https://doi.org/10.1364/OE.20.005817, http://www.opticsexpress.org/abstract.cfm?URI=oe-20-6-5817

  34. Richardson, W.H.: Bayesian-based iterative method of image restoration. J. Opt. Soc. Am. 62(1), 55–59 (1972). https://doi.org/10.1364/JOSA.62.000055, http://www.osapublishing.org/abstract.cfm?URI=josa-62-1-55

  35. Shang, C., Yang, F., Huang, D., Lyu, W.: Data-driven soft sensor development based on deep learning technique. J. Process Control 24(3), 223–233 (2014)

    Article  Google Scholar 

  36. Shen, D., Wu, G., Suk, H.I.: Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 19, 221–248 (2017)

    Article  Google Scholar 

  37. Song, Z., et al.: Temporal and spatial variability of water status in plant leaves by terahertz imaging. IEEE Trans. Terahertz Sci. Technol. 8, 520–527 (2018)

    Article  Google Scholar 

  38. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  39. Tao, X., Gao, H., Wang, Y., Shen, X., Wang, J., Jia, J.: Scale-recurrent network for deep image deblurring (2018)

    Google Scholar 

  40. Tepe, J., Schuster, T., Littau, B.: A modified algebraic reconstruction technique taking refraction into account with an application in terahertz tomography. Inverse Probl. Sci. Eng. 25(10), 1448–1473 (2017). https://doi.org/10.1080/17415977.2016.1267168, https://doi.org/10.1080/17415977.2016.1267168

  41. Voulodimos, A., Doulamis, N., Doulamis, A., Protopapadakis, E.: Deep learning for computer vision: a brief review. Comput. Intell. Neurosci. 2018, 1–13 (2018). https://doi.org/10.1155/2018/7068349

    Article  Google Scholar 

  42. Xu, L., Ren, J.S.J., Liu, C., Jia, J.: Deep Convolutional Neural Network for Image Deconvolution. Adv. Neural Inf. Process. Syst. 27, 1790–1798. Curran Associates, Inc. (2014). http://papers.nips.cc/paper/5485-deep-convolutional-neural-network-for-image-deconvolution.pdf

  43. Xu, L.M., Fan, W., Liu, J.: High-resolution reconstruction for terahertz imaging. Appl. Opt. 53 (2014). https://doi.org/10.1364/AO.53.007891

  44. Zhang, J., et al.: Dynamic scene deblurring using spatially variant recurrent neural networks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2521–2529 (2018). https://doi.org/10.1109/CVPR.2018.00267

  45. Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. (2017). https://doi.org/10.1109/TIP.2017.2662206

  46. Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep cnn denoiser prior for image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3929–3938 (2017)

    Google Scholar 

  47. Zhuang, L., Bioucas-Dias, J.M.: Fast hyperspectral image denoising and inpainting based on low-rank and sparse representations. IEEE J. Sel. Topics Appl. Earth Obs. Remote Sens. 11(3), 730–742 (2018). https://doi.org/10.1109/JSTARS.2018.2796570

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Cultural Heritage Technology, Italian Institute of Technology, Venice, Italy

    Marina Ljubenović

  2. imec-Vision Lab, Department of Physics, University of Antwerp, Antwerp, Belgium

    Shabab Bazrafkan, Pavel Paramonov, Jan De Beenhouwer & Jan Sijbers

Authors
  1. Marina Ljubenović
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shabab Bazrafkan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pavel Paramonov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jan De Beenhouwer
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jan Sijbers
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marina Ljubenović .

Editor information

Editors and Affiliations

  1. IRISA, University of Rennes 1, Rennes, France

    Kadi Bouatouch

  2. Universidade do Porto, Porto, Portugal

    A. Augusto de Sousa

  3. University of Genova, Genova, Italy

    Manuela Chessa

  4. Mines ParisTech, Paris, France

    Alexis Paljic

  5. Linnaeus University, Växjö, Sweden

    Andreas Kerren

  6. French Civil Aviation University (ENAC), Toulouse, France

    Christophe Hurter

  7. Università di Catania, Catania, Italy

    Giovanni Maria Farinella

  8. Universitat de Barcelona, Barcelona, Spain

    Petia Radeva

  9. Escola Superior de Tecnologia de Setúbal, Setúbal, Portugal

    Jose Braz

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ljubenović, M., Bazrafkan, S., Paramonov, P., Beenhouwer, J.D., Sijbers, J. (2022). CNN-Based Deblurring of THz Time-Domain Images. In: Bouatouch, K., et al. Computer Vision, Imaging and Computer Graphics Theory and Applications. VISIGRAPP 2020. Communications in Computer and Information Science, vol 1474. Springer, Cham. https://doi.org/10.1007/978-3-030-94893-1_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-94893-1_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-94892-4

  • Online ISBN: 978-3-030-94893-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation