Efficient hybrid search for visual reconstruction problems

https://doi.org/10.1016/S0262-8856(98)00088-2Get rights and content

Abstract

Visual reconstruction refers to extracting stable descriptions from visual data ([1]; A. Blake and A. Zisserman, Visual Reconstruction, MIT Press, Cambridge, MA, 1987). Visual reconstruction problems are commonly formulated in an optimization framework and normally require the optimization of nonconvex functions especially when discontinuity preserving image/shape recovery is the goal. Example problems include, image restoration, surface reconstruction, shape from shading etc. Most existing deterministic methods fail to reach the global optimum and lack the generality to incorporate reasonably complex interactions between boolean valued line process variables used for representing the presence (absence) of discontinuities. Stochastic methods for solving such problems e.g. the simulated annealing algorithm or variants thereof do achieve a global optimum but are plagued by slow convergence rates.

In this paper, we present a new hybrid search algorithm as an efficient solution for achieving the global optimum of the nonconvex function derived from a Markov random field formulation which allows for incorporation of complex interactions between the line process variables to better constraint the line processes. In the hybrid search, for the stochastic part, we develop an informed genetic algorithm (GA) while employing an incomplete Cholesky preconditioned conjugate gradient algorithm ([23]; S.H. Lai and B.C. Vemuri, Robust and efficient algorithms for optical flow computation, in: Proceedings of the International Symposium on Computer Vision, Coral Gables, FL, 1995, pp. 455–460) for the deterministic part. Our informed GA consists of a reproduction operator and an informed mutation operator. The informed mutation operator exploits specific domain knowledge in the search and is accomplished by the Gibbs sampler. Our hybrid search algorithm is highly parallelizable and leads to a globally optimal solution. The performance of our algorithm is demonstrated via experimental results on the sparse data surface reconstruction and the image restoration problem.

Introduction

Visual reconstruction refers to generating stable descriptions from visual data [1]. In their seminal work, Blake and Zisserman [1] define, a stable description as any description that is invariant to a class of distortions that characterize the image formation process e.g., optical blurring, sensor noise, rotation and translation in the image plane etc. Example tasks of visual reconstruction include image restoration, surface reconstruction, optical flow computation, shape from shading, stereo matching, the lightness problem [2], [3], [4], [5], [6], [7], [8] and others.

Visual reconstruction problems may be formulated either in a deterministic or a probabilistic framework. A popular technique in the deterministic framework is based on the regularization theory [9], [6], [10] and leads to the minimization of energy functionals. In the probabilistic formulation, a Markov random field (MRF) model is used to characterize the function being estimated and the prior distribution of this MRF model can be identified with the smoothness constraints in the regularization formulation. Both the regularization formulation and the MRF formulation of visual reconstruction problems lead to the minimization of equivalent energy functions [6], [10].

Discontinuities in images contain very crucial information for deriving high level image representations. Thus, numerous algorithms have been developed for discontinuity preserving visual reconstruction and there is abundant literature. Early work on visual reconstruction problems reported in literature ignores discontinuities altogether [4], [2], [11], which yields undesirable smoothing over discontinuity locations. However, later problem formulations allowed for incorporation of prespecified discontinuities [12], [13], [14], [15]. An elegant way to treat discontinuities is via the introduction of a set of binary variables called the line processes. Each line process variable takes a value of one in the presence of discontinuities and zero otherwise. In the past, line processes have been included in both the deterministic [16], [1] and the MRF [17] formulations. The resulting energy function involves real variables (for surface or image) coupled with binary variables (for discontinuities) and is therefore a nonconvex function. Both formulations lead to problems involving the minimization of nonconvex functions.

Numerous algorithms have been proposed to solve this coupled nonconvex minimization problem. They can be categorized into the deterministic methods [1], [18], [19] and the stochastic methods [17], [20]. Most of the deterministic methods can only deal with simple constraints on the line process [1], [19], [21]; in addition, they can not find the global minimum solution. Stochastic optimization methods for achieving the global optimum solution consist primarily of the simulated annealing (SA) type algorithms. Simulated annealing is a general global optimization algorithm with a slow convergence rate making it impractical for many applications involving large size problems.

The deterministic methods for solving the coupled non-convex minimization in the visual reconstruction problems have attracted more attention than the stochastic methods, since they usually converge faster and are more practical to use. However, most deterministic methods are restricted to handling only simple constraints on the line processes. For example, the well-known GNC (graduated nonconvex) algorithm proposed by Blake and Zisserman [1] is applicable only for the case when there is no interaction between the individual line process variables, i.e. no smoothness constraint is imposed on the line process. Recently, Bedini et al. [22] generalized the GNC algorithm to allow simple interactions between line process variables. The new closed form line energy contains terms that reduce the probability of occurrence of discontinuities which are parallel and neighboring to each other. However, it is very difficult to generalize the GNC algorithm to include arbitrary line energies. This is because, in general, the procedure for deriving the dual energy to eliminate the line processes in the GNC algorithm will be extremely complicated. Similarly, both the mean field annealing algorithm [19] and the analog network approaches [18], [21] can only deal with specific types of constraints on the line process, for e.g. the energy term for constraining the line processes must have a simple closed-form expression. Unfortunately, it is usually very difficult to obtain a simple closed-form expression for the energy of the line process constraints in an MRF model. Therefore, these deterministic methods are in general not applicable for solving the nonconvex minimization problem with a general MRF formulation as described in this paper.

In this paper, we propose a novel hybrid search algorithm as a solution for the general coupled nonconvex minimization problem with an MRF formulation which allows for incorporation of complex interactions between the line process variables to better constraint the line processes. This hybrid search algorithm is a combination of a stochastic and a deterministic search technique. For the stochastic search, we develop a new informed genetic algorithm (GA) whereas for the deterministic search, we employ an incomplete Cholesky preconditioned conjugate gradient algorithm [23]. Our informed GA consists of a reproduction operator and an informed mutation operator which exploits specific domain knowledge in the search and is accomplished by the Gibbs sampler. We present promising experimental results of applying this algorithm to sparse (synthetic) data surface reconstruction with discontinuity detection and the image restoration problems.

The rest of the paper is organized as follows. We briefly review the MRF formulation for the visual reconstruction problems and the resulting nonconvex optimization problems in the next section. A new hybrid search algorithm which comprises of a stochastic search algorithm for the line processes and a deterministic algorithm for each smaller problem (given the line process) is proposed in Section 3. In Section 4, we present a new informed GA as the stochastic search for the line process in our hybrid search algorithm. In Section 5, we briefly present the deterministic algorithm used for solving the quadratic optimization obtained for prescribed discontinuities. Section 6 contains the implementation results for the sparse data surface reconstruction and image restoration problems using the proposed hybrid search algorithm. Finally, we conclude in Section 7.

Section snippets

Markov Random Field Formulation

In this section, we will briefly present the probabilistic formulation of a general visual reconstruction problem and use it for solving two example tasks namely, the discontinuity preserving surface reconstruction and the image restoration. The formulation leads to non-convex optimization. We choose the probabilistic formulation due to its many advantages over the deterministic counterpart. For instance, a probabilistic model can provide second (covariance) or higher order statistics of the

Proposed solution

In this section, we present the outline of a novel hybrid (stochastic+deterministic) search algorithm as an efficient solution to the coupled (binary-real) nonconvex optimization problem. This hybrid search algorithm consists of an informed genetic algorithm (GA) and an incomplete Cholesky preconditioned conjugate gradient algorithm [23]. The informed genetic algorithm is employed as a global minimizer on the binary line process. Within the GA, for each given line process configuration, the

Informed genetic algorithm

The GA is an adaptive search algorithm based on principles derived from the dynamics of natural population genetics. It has been successfully applied to combinatorial optimization, function optimization, classifier systems, structure optimization, pattern recognition, and other areas [30], [31].

A population of candidate solutions is generated in the beginning as the initial population, denoted by P(0). Then, the GA operators [31], i.e. reproduction, crossover and mutation, are applied to the

Incomplete Cholesky preconditioned conjugate gradient

In our hybrid search algorithm, the informed GA is used as a stochastic search for the binary line process only. For each line process configuration l visited by the GA, we need to compute its objective function value E(l), given in Eq. (12). This computation involves the minimization of a convex and quadratic function U(f|l,d), which can be obtained by solving a linear system with an associated symmetric positive–definite (SPD) matrix. We designed an incomplete Cholesky preconditioner for use

Experimental results

In this section, we present the experimental results of applying our hybrid search algorithm to the surface reconstruction and image restoration problems. In our experiments, our hybrid search algorithm consists of the informed GA for the binary line process l and the incomplete Cholesky preconditioned conjugate gradient algorithm for determining the fitness function value for a pre-specified line process configuration.

Discussions and conclusions

A new hybrid search algorithm was presented in this paper to solve the coupled (binary-real) nonconvex optimization problem. Our hybrid search algorithm consists of a novel informed GA and the incomplete Cholesky preconditioned conjugate gradient algorithm [23]. The informed GA is used as a stochastic global minimizer for a new energy function consisting of the binary line process variables only. Inside the GA, an incomplete Cholesky preconditioned conjugate gradient is used to determine the

Acknowledgements

Supported in part by the NSF grant ECS-9210648 and the Whitaker Foundation.

References (41)

  • T. Simchony et al.

    Direct analytical methods for solving Poisson equations in computer vision problems

    IEEE Trans. Patt. Anal. Mach. Intell.

    (1990)
  • A.N. Tikhonov, V.Y. Arsenin, Solutions of Ill-posed Problems, Winston, Washington, DC,...
  • D. Terzopoulos

    Regularization of inverse visual problems involving discontinuities

    IEEE Trans. Patt. Anal. Mach. Intell.

    (1986)
  • T. Poggio, V. Torre, Ill-posed problems and regularization analysis in early vision, Technical Report AI Memo 773, MIT,...
  • D. Terzopoulos

    The computation of visible representations

    IEEE Trans. Patt. Anal. Mach. Intell.

    (1988)
  • R. Szeliski

    Fast surface interpolation using hierarchical basis functions

    IEEE Trans. Patt. Anal. Mach. Intell.

    (1990)
  • S.H. Lai, B.C. Vemuri, An O(N) iterative solution to the Poisson equations in low-level vision problems, in Proc. IEEE...
  • S.H. Lai et al.

    Physically-based adaptive preconditioning for early vision

    IEEE Trans. Patt. Anal. Mach. Intell.

    (1997)
  • D. Mumford, J. Shah, Boundary detection by minimizing functionals, in: Proc. IEEE Conf. Comput. Vis. Patt. Recog., San...
  • S. Geman et al.

    Stochastic relaximation, Gibbs distributions, and the Bayesian restoration of images

    IEEE Trans. Patt. Anal. Mach. Intell.

    (1984)
  • Cited by (13)

    • Preconditioned edge-preserving image deblurring and denoising

      2001, Pattern Recognition Letters
      Citation Excerpt :

      A paper by Fessler and Booth (1999) considers the case of implicitly addressed discontinuity and space-variant imaging systems, and proposes another kind of preconditioner which performs better than the circulant type, since the structure of the problem is not block-Toeplitz. Another work, by Lai and Vemuri (1999), proposes a strategy similar to ours, still for image deconvolution and explicit lines. In particular, the authors present a hybrid algorithm where the stochastic part is performed by a genetic algorithm which exploits specific domain knowledge in the search (informed genetic algorithm), while the deterministic part employs an incomplete Cholesky PCG algorithm.

    • Image Analysis and Computer Vision: 1999

      2000, Computer Vision and Image Understanding
    • Vision and evolution: State of the art

      2016, Natural Computing Series
    • Proposal of a cellular neural network for motion estimation in camera space for applications with industrial robots

      2008, CISCI 2008 - Septima Conferencia Iberoamericana en Sistema, Cibernetica e Informatica 5to SIECI 2008, 3er Simposium Internacional en Comunicacion del Conocimiento y Conferencias, CCC 2008 - Memorias
    • Evolutionary computer vision

      2007, Proceedings of GECCO 2007: Genetic and Evolutionary Computation Conference, Companion Material
    View all citing articles on Scopus
    View full text