Skip to main content
SpringerLink
Log in

Backward-link computational imaging using batch learning networks

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Optical images are inevitably stained by the multiple aberrations from an optical imaging link itself (OILI, e.g., the camera itself involving an optical system, an electronic system, and a sensor). The development of techniques that enable optical full-link imaging without the OILI aberrations is desirable. In this paper, we demonstrate an aberration-free backward-link imaging method for optical full-link imaging. This method utilizes a batch learning network to reconstruct the invariable point spread function (IPSF) of the imaging processing, which is constant and produced by the design, fabrication, and assembly. The inverted IPSF in conjunction with a digital filter can correct the distorted wavefront aberrations caused by the OILI. This is confirmed experimentally. This method opens the door to aberration-free full-link optical imaging for the involved imaging system applications, such as Coarse Holography Displays.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22:1330–1334

    Google Scholar 

  2. Pohl C, Van Genderen JL (1998) Review article multisensor image fusion in remote sensing: concepts, methods and applications. Int J Remote Sens 19(5):823–854

    Google Scholar 

  3. Hirano A, Welch R, Lang H (2003) Mapping from ASTER stereo image data: DEM validation and accuracy assessment. ISPRS J Photogramm Remote Sens 57(5):356–370

    Google Scholar 

  4. Li J, Xing F, You Z (2014) Compression of multispectral images with comparatively few bands using posttransform Tucker decomposition. Math Probl Eng 2014:1–17

    Google Scholar 

  5. Freedman WL, Madore BF, Gibson BK, Ferrarese L, Kelson DD, Sakai S, Mould JR, Kennicutt RC Jr., Ford HC, Graham JA, Huchra JP (2001) Final results from the Hubble Space Telescope key project to measure the Hubble constant. Astrophys J 553(1):47

    Google Scholar 

  6. Li J, Smithwick Q, Chu D (2018) Full bandwidth dynamic coarse integral holographic displays with large field of view using a large resonant scanner and a galvanometer scanner. Opt Express 26(13):17459–17476

    Google Scholar 

  7. Li J, Xing F, Chu D, Liu Z (2016) High-accuracy self-calibration for smart, optical orbiting payloads integrated with attitude and position determination. Sensors 16(8):1176

    Google Scholar 

  8. Li J, Xing F, Shrestha PK, Shi F, Liu Z (2019) On-orbit self-focusing using conjugated optical fiber waveguides for space optical cameras. IEEE Geosci Remote Sens Lett 16:1703–1705

    Google Scholar 

  9. Rome M, Fleck HG, Hines DC (1964) The quadrant multiplier phototube, a new star-tracker sensor. Appl Opt 3(6):691–695

    Google Scholar 

  10. Sun T, Xing F, Wang X, Li J, Wei M, You Z (2016) Effective star tracking method based on optical flow analysis for stellar cameras. Appl Opt 55(36):10335–10340

    Google Scholar 

  11. Yadid-Pecht O, Pain B, Staller C, Clark C, Fossum E (1997) CMOS active pixel sensor star tracker with regional electronic shutter. IEEE J Solid-State Circuits 32(2):285–288

    Google Scholar 

  12. Rousseau GLA, Bostel J, Mazari B (2005) Star recognition algorithm for APS star tracker: oriented triangles. IEEE Aerosp Electron Syst Mag 20(2):27–31

    Google Scholar 

  13. Li J, Xing F, Sun T, You Z (2015) Efficient assessment method of on-board modulation transfer function of optical remote sensing sensors. Opt Express 23:6187–6208

    Google Scholar 

  14. Hwang H, Choi YW, Kwak S, Kim M, Park W (2008) MTF assessment of high resolution satellite images using ISO 12233 slanted-edge method. In: Image and signal processing for remote sensing XIV. International Society for Optics and Photonics, vol 7109, p 710905

  15. Oh E, Choi JK (2014) GOCI image enhancement using an MTF compensation technique for coastal water applications. Opt Express 22(22):26908–26918

    Google Scholar 

  16. Lee J, Chun J, Lee D (2012) MTF compensation algorithm based on blind deconvolution for high-resolution remote sensing satellite. In: Visual information processing XXI. International Society for Optics and Photonics, vol 8399, p 83990R

  17. Blonski S, Pagnutti MA, Ryan R, Zanoni V (2002) In-flight edge response measurements for high-spatial-resolution remote sensing systems. In: Earth observing systems VII. International Society for Optics and Photonics, vol 4814, pp 317–327

  18. Guo XJ, Liu XL, Ni C, Liu B, Huang SM, Gu M (2011) Improving image quality of x-ray in-line phase contrast imaging using an image restoration method. Opt Express 19(23):23460–23468

    Google Scholar 

  19. Oh E, Ahn KB, Cho S, Ryu JH (2013) A modulation transfer function compensation for the geostationary ocean color imager (GOCI) based on the wiener filter. J Astron Space Sci 30(4):321–326

    Google Scholar 

  20. Mee JWS (2010) Modulation transfer function compensation through a modified Wiener filter for spatial image quality improvement. Sel Top Power Syst Remote Sens, 177–182

  21. Kundur D, Hatzinakos D (1996) Blind image deconvolution. IEEE Signal Process Mag 13(3):43–64

    Google Scholar 

  22. Chan TF, Wong CK (1998) Total variation blind deconvolution. IEEE Trans Image Process 7(3):370–375

    Google Scholar 

  23. Shalvi O, Weinstein E (1990) New criteria for blind deconvolution of nonminimum phase systems (channels). IEEE Trans Inf Theory 36(2):312–321

    MATH  Google Scholar 

  24. Krishnan D, Tay T, Fergus R (2011) Blind deconvolution using a normalized sparsity measure. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR), pp 233–240

  25. Cannon M (1976) Blind deconvolution of spatially invariant image blurs with phase. IEEE Trans Acoust Speech Signal Process 24(1):58–63

    Google Scholar 

  26. Amari SI, Douglas SC, Cichocki A, Yang HH (1997) Multichannel blind deconvolution and equalization using the natural gradient. In: First IEEE signal processing workshop on signal processing advances in wireless communications, pp 101–104

  27. Fish DA, Brinicombe AM, Pike ER, Walker JG (1995) Blind deconvolution by means of the Richardson–Lucy algorithm. JOSA A 12(1):58–65

    Google Scholar 

  28. Levin A, Weiss Y, Durand F, Freeman WT (2011) Efficient marginal likelihood optimization in blind deconvolution. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR), pp 2657–2664

  29. Schulz TJ (1993) Multiframe blind deconvolution of astronomical images. JOSA A 10(5):1064–1073

    Google Scholar 

  30. Holmes TJ (1992) Blind deconvolution of quantum-limited incoherent imagery: maximum-likelihood approach. JOSA A 9(7):1052–1061

    Google Scholar 

  31. Liang J, Williams DR, Miller DT (1997) Supernormal vision and high-resolution retinal imaging through adaptive optics. JOSA A 14(11):2884–2892

    Google Scholar 

  32. Zawadzki RJ, Jones SM, Olivier SS, Zhao M, Bower BA, Izatt JA, Choi S, Laut S, Werner JS (2005) Adaptive-optics optical coherence tomography for high-resolution and high-speed 3D retinal in vivo imaging. Opt Express 13(21):8532–8546

    Google Scholar 

  33. Angel JRP (1994) Ground-based imaging of extrasolar planets using adaptive optics. Nature 368(6468):203

    Google Scholar 

  34. Ragazzoni R, Marchetti E, Valente G (2000) Adaptive-optics corrections available for the whole sky. Nature 403(6765):54

    Google Scholar 

  35. Ji N, Milkie DE, Betzig E (2009) Adaptive optics via pupil segmentation for high-resolution imaging in biological tissues. Nat Methods 7(2):141

    Google Scholar 

  36. Angel JRP, Wizinowich P, Lloyd-Hart M, Sandler D (1990) Adaptive optics for array telescopes using neural-network techniques. Nature 348(6298):221

    Google Scholar 

  37. Macintosh B, Troy M, Doyon R, Graham J, Baker K, Bauman B, Marois C, Palmer D, Phillion D, Poyneer L, Crossfield I (2006) Extreme adaptive optics for the Thirty Meter Telescope. In: Advances in adaptive optics II, international society for optics and photonics, vol 6272, p 62720 N

  38. Roorda A, Romero-Borja F, Donnelly WJ III, Queener H, Hebert TJ, Campbell MC (2002) Adaptive optics scanning laser ophthalmoscopy. Opt Express 10(9):405–412

    Google Scholar 

  39. Sandler DG, Barrett TK, Palmer DA, Fugate RQ, Wild WJ (1991) Use of a neural network to control an adaptive optics system for an astronomical telescope. Nature 351(6324):300

    Google Scholar 

  40. Ohm JR (1994) Three-dimensional subband coding with motion compensation. IEEE Trans Image Process 3(5):559–571

    Google Scholar 

  41. Nakaya Y, Harashima H (1994) Motion compensation based on spatial transformations. IEEE Trans Circuits Syst Video Technol 4(3):339–356

    Google Scholar 

  42. Lareau AG (1993) Electro-optical imaging array with motion compensation. In: International society for optics and photonics in airborne reconnaissance XVII, vol 2023, pp 65–80

  43. Garivait S, Quisefit JP, De Chateaubourg P, Malingre G (1997) Multi-element analysis of plants by WDXRF using the scattered radiation correction method. X-Ray Spectrom Int J 26(5):257–264

    Google Scholar 

  44. Hollands KGT, Crha SJ (1987) An improved model for diffuse radiation: correction for atmospheric back-scattering. Sol Energy 38(4):233–236

    Google Scholar 

  45. Holst GC (2011) Imaging system fundamentals. Opt Eng 50(5):052601

    Google Scholar 

  46. Viallefont-Robinet F, Helder D, Fraisse R, Newbury A, van den Bergh F, Lee D, Saunier S (2018) Comparison of MTF measurements using edge method: towards reference data set. Opt Express 26(26):33625–33648

    Google Scholar 

  47. Viallefont-Robinet F, Léger D (2010) Improvement of the edge method for on-orbit MTF measurement. Opt Express 18(4):3531–3545

    Google Scholar 

  48. Storey JC (2001) Landsat 7 on-orbit modulation transfer function estimation. SPIE 4540:50–62

    Google Scholar 

  49. Goldstein T, Stanley O (2009) The split Bregman method for L1-regularized problems. SIAM J Imaging Sci 2(2):323–343

    MathSciNet  MATH  Google Scholar 

  50. Goldstein T, Bresson X, Osher S (2010) Geometric applications of the split Bregman method: segmentation and surface reconstruction. J Sci Comput 45(1–3):272–293

    MathSciNet  MATH  Google Scholar 

  51. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Google Scholar 

  52. Chen GH, Yang CL, Xie SL (2006) Gradient-based structural similarity for image quality assessment. In: 2006 IEEE international conference on image processing, pp 2929–2932

  53. Mattia C, De Vendictis L (2009) A procedure for high resolution satellite imagery quality assessment. Sensors 9(5):3289–3313

    Google Scholar 

  54. Li J, Liu Z (2019) High-resolution dynamic inversion imaging with motion-aberrations-free using optical flow learning networks. Sci Rep 9(1):1–12

    Google Scholar 

  55. Wu C, Tai XC (2010) Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models. SIAM J Imaging Sci 3(3):300–339

    MathSciNet  MATH  Google Scholar 

  56. Boyd S, Parikh N, Chu E, Peleato B, Eckstein J (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends® Mach Learn 3(1):1–122

    MATH  Google Scholar 

  57. Daneshmand A, Facchinei F, Kungurtsev V, Scutari G (2015) Hybrid random/deterministic parallel algorithms for convex and nonconvex big data optimization. IEEE Trans Signal Process 63(15):3914–3929

    MathSciNet  MATH  Google Scholar 

  58. Daneshmand A, Facchinei F, Kungurtsev V, Scutari G (2014) Flexible selective parallel algorithms for big data optimization. In 2014 48th Asilomar conference on signals, systems and computers, pp 3–7. IEEE

  59. Dassios I, Fountoulakis K, Gondzio J (2015) A preconditioner for a primal-dual newton conjugate gradients method for compressed sensing problems. SIAM J Sci Comput 37:A2783–A2812

    MathSciNet  MATH  Google Scholar 

  60. Dassios I, Fountoulakis K, Gondzio J (2014) A second-order method for compressed sensing problems with coherent and redundant dictionaries. arXiv preprint arXiv:1405.4146

  61. Dassios I (2019) Analytic loss minimization: theoretical framework of a second order optimization method. Symmetry MDPI 11(2):136

    MATH  Google Scholar 

  62. Ni T, Zhai J (2016) A matrix-free smoothing algorithm for large-scale support vector machines. Inf Sci 358:29–43

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Photonics and Sensors Group, Department of Engineering, University of Cambridge, 9 JJ Thomson Avenue, Cambridge, CB3 0FA, UK

    Jin Li

  2. Department of Precision Instrument, Tsinghua University, Beijing, 100084, China

    Fei Xing

  3. State Key Laboratory of Precision Measurement Technology and Instruments, Tsinghua University, Beijing, 100084, China

    Fei Xing

  4. Department of Electronics and Information Engineering, Changchun University Science and Technology, Jinlin, 130022, China

    Yanyan Liu

  5. Division of Optics, National Institute of Metrology, Beijing, 100029, China

    Zilong Liu

Authors
  1. Jin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fei Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanyan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zilong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fei Xing.

Ethics declarations

Conflict of interest

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by the National Key Research and Development Program of China under Grant 2016YFB0501201 and National Key Research and Development Plan of China (No. 2017YFF0205103).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, J., Xing, F., Liu, Y. et al. Backward-link computational imaging using batch learning networks. Neural Comput & Applic 32, 12895–12907 (2020). https://doi.org/10.1007/s00521-020-04734-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-04734-9

Keywords

Search

Navigation