Skip to main content
SpringerLink
Log in

Convergence properties of Gauss-Newton iterative algorithms in nonlinear image restoration

  • Published:
Multidimensional Systems and Signal Processing Aims and scope Submit manuscript

Abstract

Nonlinear image restoration is a complicated problem that is receiving increasing attention. Since every image formation system involves a built-in nonlinearity, nonlinear image restoration finds applications in a wide variety of research areas. Iterative algorithms have been well established in the corresponding linear restoration problem. In this paper, a generalized analysis regarding the convergence properties of nonlinear iterative algorithms is introduced. Moreover, the applications of the iterative Gauss-Newton (GN) algorithm in nonlinear image restoration are considered. The convergence properties of a general class of nonlinear iterative algorithms are rigorously studied through the Global Convergence Theorem (GCT). The derivation of the convergence properties is based on the eigen-analysis, rather than on the norm analysis. This approach offers a global picture of the evolution and the convergence properties of an iterative algorithm. Moreover, the generalized convergence-analysis introduced may be interpreted as a link towards the integration of minimization and projection algorithms. The iterative GN algorithm for the solution of the least-squares optimization problem is introduced. The computational complexity of this algorithm is enormous, making its implementation very difficult in practical applications. Structural modifications are introduced, which drastically reduce the computational complexity while preserving the convergence rate of the GN algorithm. With the structural modifications, the GN algorithm becomes particularly useful in nonlinear optimization problems. The convergence properties of the algorithms introduced are readily derived, on the basis of the generalized analysis through the GCT. The application of these algorithms on practical problems, is demonstrated through several examples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • H.C. Andrews and B.R. Hunt,Digital Image Restoration, Prentice-Hall, Englewood Cliffs, NJ: 1977.

    Google Scholar 

  • H.J. Trussell, “A Priori Knowledge in Algebraic Reconstruction Methods,” inAdvances in Computer Vision and Image Processing, vol.1, (T.S. Huang ed.) JAI Press Inc., Greenwich, Conneticut, 1984.

    Google Scholar 

  • M. Bertero, T.A. Poggio, and V. Torre, “Ill-Posed Problems in Early Vision,”IEEE Proceedings, vol. 76, no. 8, 1988.

  • B.R. Hunt, “The Application of Constrained Least Squares Estimation to Image Restoration by Digital Computer,”IEEE Trans. on Computers, vol. C-22, no. 9, 1973.

  • B.R. Hunt, “Bayesian Methods in Nonlinear Digital Image Restoration,”IEEE Trans. on Computers, vol. C-26, no. 3, 1977.

  • D.T. Kuan, A.A. Sawchuk, T.C. Strand, and P. Chavel, “Adaptive Noise Smoothing Filter for Images with Signal Dependent Noise,”IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. PAMI-7, no. 2, 1985.

  • J.S. Lee, “Speckle Analysis and Smoothing of Synthetic Aperture Radar Images,”Computer Graphics and Image Processing,” vol. 17, pp. 24–32, 1981.

    Google Scholar 

  • R. Bernstein, “Adaptive Nonlinear Filters for Simultaneous Removal of Different Kinds of Noise in Images,”IEEE Trans. on Circuits and Systems, vol. CAS-34, no. 11, 1987.

  • V.S. Frost, J.A. Stiles, K.S. Shanmugam, J.C. Holtzman, and S.A. Smith, “An Adaptive Filter for Smoothing Noisy Radar Images,”Proceedings of the IEEE, vol. 69, no. 1, 1981.

  • S. German, and D. Geman, “Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images,”IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. PAMI-6, no. 6, 1984.

  • H.J. Trussell and B.R. Hunt, “Improved Methods of Maximum a Posteriori Restoration,”IEEE Trans. on Computers, vol. C-27, no. 1, 1979.

  • M.E. Zervakis and A.N. Venetsanopoulos, “A Generalized Adaptive Model for Nonlinear Image Restoration,”Proc. of Int'l Symposium on Circuits and Systems ISCAS' 89, Portland, Oregon, May 9–11, 1989, July 1991.

  • M.E. Zervakis and A.N. Venetsanopoulos, “Iterative Least Squares Estimators in Nonlinear Image Restoration,”IEEE Trans. on Acoustics, Speech, and Signal Processing, in Press, April 1992.

  • D.G. Luenberger,Introduction to Linear and Nonlinear Programming, Addison-Wesley: Reading, MA 1973.

    Google Scholar 

  • R.W. Schafer, R.M. Mersereau, and M.A. Richards, “Constrainted Iterative Restoration Algorithms,”Proceedings of the IEEE, vol. 69, no. 4, 1981.

  • V.T. Tom, T.F. Quatieri, M.H. Hayes, and J.H. McClellan, “Convergence of Iterative Nonexpansive Signal Reconstruction Algorithms,”IEEE Trans. on Acoustics, Speech, and Signal Processing, vol. ASSP-29, no. 5, 1981.

  • J.A. Gadzow, “Signal Enhancement—A Composite Property Mapping Algorithm,”IEEE Trans. on Acoustics, Speech, and Signal Processing, vol. ASSP-36, no. 1, 1988.

  • H.J. Trussell, “Convergence Criteria for Iterative Restoration Methods,”IEEE Trans. on Acoustics, Speech, and Signal Processing, vol. ASSP-31, no. 1, 1983.

  • H.J. Trussell and M.R. Civanlar, “The Feasible Solution in Image Restoration,”IEEE Trans. on Acoustics, Speech, and Signal Processing, vol. ASSP-32, no. 2, 1984.

  • M.E. Zervakis, “Nonlinear Image Restoration Techniques,” Ph.D. Thesis, University of Toronto, 1989.

  • R.L. Lagendijk, J. Biemond, and D.E. Boekee, “Regularized Iterative Image Restoration with Ringing Reduction,”IEEE Trans. on Acoustics, Speech, and Signal Processing, vol. ASSP-36, no. 12, 1988.

  • A. Katsaggelos, “Iterative Image Restoration Algorithms,”Optical Engineering, vol. 28, no. 7, pp. 735–748, 1989.

    Google Scholar 

  • D.C. Youla, “Mathematical Theory of Image Restoration by the Method of Convex Projections,” inImage Recovery: Theory and Application, (H. Stark, ed.), Academic Press, Orlando, FL, 1987.

    Google Scholar 

  • R. Gallant,Nonlinear Statistical Models J. Wiley: New York, 1987.

    Google Scholar 

  • G.H. Golub, and C.F. Van Loan,Matrix Computations Johns Hopkins, Baltimore, MD, 1983.

    Google Scholar 

  • M.E. Zervakis and A.N. Venetsanopoulos, “Iterative algorithms with Fast Convergence Rates in Nonlinear Image Restoration,”Proc. of the 1991 SPIE/SPSE Symp. on Electronic Imaging Science and Technology, San Jose, CA, Feb. 1991.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, University of Minnesota, 271 Marshall W. Alworth Hall, 10 University Drive, 55812-2496, Duluth, MN

    M. E. Zervakis

  2. Department of Electrical Engineering, University of Toronto, 35 St. George St., M5S 1A4, Toronto, Ontario, Canada

    A. N. Venetsanopoulos

Authors
  1. M. E. Zervakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. N. Venetsanopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zervakis, M.E., Venetsanopoulos, A.N. Convergence properties of Gauss-Newton iterative algorithms in nonlinear image restoration. Multidim Syst Sign Process 2, 287–318 (1991). https://doi.org/10.1007/BF01952237

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01952237

Keywords

Search

Navigation