Skip to main content
Log in

The Dynamics of Nonlinear Relaxation Labeling Processes

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

We present some new results which definitively explain thebehavior of the classical, heuristic nonlinear relaxation labelingalgorithm of Rosenfeld, Hummel, and Zucker in terms of theHummel-Zucker consistency theory and dynamical systems theory. Inparticular, it is shown that, when a certain symmetry condition is met,the algorithm possesses a Liapunov function which turns out to be (thenegative of) a well-known consistency measure. This follows almostimmediately from a powerful result of Baum and Eagon developed in thecontext of Markov chain theory. Moreover, it is seen that most of theessential dynamical properties of the algorithm are retained when thesymmetry restriction is relaxed. These properties are also shown tonaturally generalize to higher-order relaxation schemes. Someapplications and implications of the presented results are finallyoutlined.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. A. Rosenfeld, R.A. Hummel, and S.W. Zucker, “Scene labeling by relaxation operations,” IEEE Trans. Syst. Man Cybern., Vol. 6, pp. 420–433, 1976.

    Google Scholar 

  2. J. Kittler and J. Illingworth, “Relaxation labeling algorithms—A review,” Image Vision Comput., Vol. 3, pp. 206–216, 1985.

    Google Scholar 

  3. S. Peleg, “A new probabilistic relaxation scheme,” IEEE Trans. Pattern Machine Intell., Vol. 2, pp. 362–369, 1980.

    Google Scholar 

  4. J. Kittler and E.R. Hancock, “Combining evidence in probabilistic relaxation,” Int. J. Pattern Recognition Artif. Intell., Vol. 3, pp. 29–51, 1989.

    Google Scholar 

  5. W.J. Christmas, J. Kittler, and M. Petrou, “Structural matching in computer vision using probabilistic relaxation,” IEEE Trans. Pattern Anal. Machine Intell., Vol. 17, pp. 749–764, 1995.

    Google Scholar 

  6. R.A. Hummel and S.W. Zucker, “On the foundations of relaxation labeling processes,” IEEE Trans. Pattern Anal. Machine Intell., Vol. 5, pp. 267–287, 1983.

    Google Scholar 

  7. A.J. Stoddart, M. Petrou, and J. Kittler, “On the foundations of probabilistic relaxation with product support,” submitted for publication, 1996.

  8. S. Ullman, “Relaxation and constrained optimization by local processes,” Comput. Graph. Image Processing,Vol. 10, pp. 115–125, 1979.

    Google Scholar 

  9. O.D. Faugeras and M. Berthod, “Improving consistency and reducing ambiguity in stochastic labeling: An optimization approach,” IEEE Trans. Pattern Anal. Machine Intell., Vol. 3, pp. 412–424, 1981.

    Google Scholar 

  10. J. Illingworth and J. Kittler, “Optimization algorithms in probabilistic relaxation labeling,” in Pattern Recognition Theory and Applications, P.A. Devijver and J. Kittler (Eds.), Springer-Verlag: Berlin, 1987, pp. 109–117.

    Google Scholar 

  11. E.R. Hancock and J. Kittler, “Discrete relaxation,” Pattern Recognition, Vol. 23, pp. 711–733, 1990.

    Google Scholar 

  12. S.A. Lloyd, “An optimization approach to relaxation labeling algorithms,” Image Vision Comput., Vol. 1, pp. 85–91, 1983.

    Google Scholar 

  13. T. Elfving and J.-O. Eklundh, “Some properties of stochastic labeling procedures,” Comput. Graph. Image Processing, Vol. 20, pp. 158–170, 1982.

    Google Scholar 

  14. M. Levy, “A new theoretical approach to relaxation—Application to edge detection,” in Proc. 9th Int. Conf. Pattern Recognition, Rome, Italy, 1988, pp. 208–212.

  15. S. Peleg and A. Rosenfeld, “Determining compatibility coefficients for curve enhancement relaxation processes,” IEEE Trans. Syst. Man Cybern., Vol. 8, pp. 548–555, 1978.

    Google Scholar 

  16. R.C. Wilson and E.R. Hancock, “A Bayesian compatibility model for graph matching,” Pattern Recognition Lett., Vol. 17, pp. 263–276, 1996.

    Google Scholar 

  17. M. Pelillo and M. Refice, “Learning compatibility coefficients for relaxation labeling processes,” IEEE Trans. Pattern Anal. Machine Intell., Vol. 16, pp. 933–945, 1994.

    Google Scholar 

  18. M. Pelillo, F. Abbattista, and A. Maffione, “An evolutionary approach to training relaxation labeling processes,” Pattern Recognition Lett., Vol. 16, pp. 1069–1078, 1995.

    Google Scholar 

  19. M. Pelillo and A.M. Fanelli, “Autoassociative learning in relaxation labeling networks,” in Proc. 13th Int. Conf. Pattern Recognition, Vienna, Austria, 1996.

  20. R.L. Kirby, “A product rule relaxation method,” Comput. Graph. Image Processing, Vol. 13, pp. 158–189, 1980.

    Google Scholar 

  21. E.R. Hancock and J. Kittler, “Edge-labeling using dictionary-based relaxation,”IEEE Trans. Pattern Anal. Machine Intell., Vol. 12, pp. 165–181, 1990.

    Google Scholar 

  22. S.W. Zucker, E.V. Krishnamurthy, and R.L. Haar, “Relaxation processes for scene labeling: Convergence, speed, and stability,” IEEE Trans. Syst. Man Cybern., Vol. 8, pp. 41–48, 1978.

    Google Scholar 

  23. X. Zhuang, R.M. Haralick, and H. Joo, “A simplex-like algorithm for the relaxation labeling process,” IEEE Trans. Pattern Anal. Machine Intell., Vol. 11, pp. 1316–1321, 1989.

    Google Scholar 

  24. L.E. Baum and J.A. Eagon, “An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology,” Bull. Amer. Math. Soc., Vol. 73, pp. 360–363, 1967.

    Google Scholar 

  25. G.R. Blakley, “Homogeneous nonnegative symmetric quadratic transformations,” Bull. Amer. Math. Soc., Vol. 70, pp. 712–715, 1964.

    Google Scholar 

  26. L.E. Baum and G.R. Sell, “Growth transformations for functions on manifolds,” Pacific J. Math., Vol. 27, pp. 211–227, 1968.

    Google Scholar 

  27. J.L. Mohammed, R.A. Hummel, and S.W. Zucker, “A gradient projection algorithm for relaxation methods,” IEEE Trans. Pattern Anal. Machine Intell., Vol. 5, pp. 330–332, 1983.

    Google Scholar 

  28. L.E. Baum, T. Petrie, G. Soules, and N. Weiss, “A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains,” Ann. Math. Statist., Vol. 41, pp. 164–171, 1970.

    Google Scholar 

  29. S.E. Levinson, L.R. Rabiner, and M.M. Sondhi, “An introduction to the application of the theory of probabilistic functions of a Markov process to automatic speech recognition,” Bell. Syst. Tech. J., Vol. 62, pp. 1035–1074, 1983.

    Google Scholar 

  30. P. Stebe, “Invariant functions of an iterative process for maximization of a polynomial,” Pacific J. Math., Vol. 43, pp. 765–783, 1972.

    Google Scholar 

  31. D.S. Passman, “The Jacobian of a growth transformation,” Pacific J. Math., Vol. 44, pp. 281–290, 1973.

    Google Scholar 

  32. P.S. Gopalakrishnan, D. Kanevsky, A. Nádas, and D. Nahamoo, “An inequality for rational functions with applications to some statistical estimation problems,” IEEE Trans. Inform. Theory, Vol. 37, pp. 107–113, 1991.

    Google Scholar 

  33. D.G. Luenberger, Introduction to Dynamic Systems, Wiley: New York, 1979.

    Google Scholar 

  34. J.P. LaSalle, The Stability and Control of Discrete Processes, Springer-Verlag: New York, 1986.

    Google Scholar 

  35. S.W. Zucker, A. Dobbins, and L. Iverson, “Two stages of curve detection suggest two styles of visual computation,” Neural Computat., Vol. 1, pp. 68–81, 1989.

    Google Scholar 

  36. G. Zoutendijk, Mathematical Programming Methods, North-Holland: Amsterdam, 1976.

    Google Scholar 

  37. P. Parent and S.W. Zucker, “Radial projection: An efficient update rule for relaxation labeling,” IEEE Trans. Pattern Anal. Machine Intell., Vol. 11, pp. 886–889, 1989.

    Google Scholar 

  38. A.M. Ostrowski, Solution of Equations and Systems of Equations, Academic Press: New York, 1966.

    Google Scholar 

  39. J.-O. Eklundh and A. Rosenfeld, “Some relaxation experiments using triples of pixels,” IEEE Trans. Syst. Man Cybern., Vol. 10, pp. 150–153, 1980.

    Google Scholar 

  40. S. Peleg, “Ambiguity reduction in handwriting with ambiguous segmentation and uncertain interpretation,” Comput. Graph. Image Processing, Vol. 10, pp. 235–245, 1979.

    Google Scholar 

  41. J. Kittler and E.R. Hancock, “Contextual decision rule for region analysis,” Image Vision Comput., Vol. 5, pp. 145–154, 1987.

    Google Scholar 

  42. A.M. Finch, R.C. Wilson, and E.R. Hancock, “Matching Delaunay triangulations by relaxation labeling,” in Computer Analysis of Images and Patterns, V. Hlavác and R. Sára (Eds.), Springer: Berlin, 1995, pp. 350–358.

    Google Scholar 

  43. C.L. Giles and T. Maxwell, “Learning, invariance, and generalization in high-order neural networks,” Appl. Optics, Vol. 26, pp. 4972–4978, 1987.

    Google Scholar 

  44. D. Psaltis and C.H. Park, “Nonlinear discriminant functions and associative memories,” in Neural Networks for Computing, J.S. Denker (Ed.), American Institute of Physics: New York, 1986, pp. 370–375.

    Google Scholar 

  45. R. Durbin and D.E. Rumelhart, “Product units: A computationally powerful and biologically plausible extension to backpropagation networks,” Neural Computat., Vol. 1, pp. 133–142, 1989.

    Google Scholar 

  46. J.J. Hopfield and D.W. Tank, “Neural computation of decisions in optimization problems,” Biol. Cybern., Vol. 52, pp. 141–152, 1985.

    Google Scholar 

  47. D.E. Van den Bout and T.K. Miller, “A traveling salesman objective function that works,” in Proc. IEEE Int. Conf. Neural Networks, San Diego, CA, 1988, pp. 299–303.

  48. M. Pelillo, “Relaxation labeling processes for the traveling salesman problem,” in Proc. Int. J. Conf. Neural Networks, Nagoya, Japan, 1993, pp. 2429–2432.

  49. M. Pelillo, “Relaxation labeling networks for the maximum clique problem,” J. Artif. Neural Networks, Special issue on “Neural Networks for Optimization,” Vol. 2, pp. 313–328, 1995.

    Google Scholar 

  50. G.V. Wilson and G.S. Pawley, “On the stability of the traveling salesman problem algorithm of Hopfield and Tank,” Biol. Cybern., Vol. 58, pp. 63–70, 1988.

    Google Scholar 

  51. T. Poggio, V. Torre, and C. Koch, “Computational vision and regularization theory,” Nature, Vol. 317, pp. 314–319, 1985.

    Google Scholar 

  52. N.M. Nasrabadi and C.Y. Choo, “Hopfield network for stereo vision correspondence,” IEEE Trans. Neural Networks, Vol. 3, pp. 5–13, 1992.

    Google Scholar 

  53. J.J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc. Natl. Acad. Sci. USA, Vol. 79, pp. 2554–2558, 1982.

    Google Scholar 

  54. D. Valentin, H. Abdi, A.J. O’Toole, and G.W. Cottrell, “Connectionist models of face processing: A survey,” Pattern Recognition, Vol. 27, pp. 1209–1230, 1994.

    Google Scholar 

  55. H. Wechsler, “Network representations and match filters for invariant object recognition,” in Pattern Recognition Theory and Applications, P.A. Devijver and J. Kittler (Eds.), Springer-Verlag: Berlin, 1987, pp. 269–276.

    Google Scholar 

  56. M. Pelillo and A.M. Fanelli, “An asymmetric associative memory model based on relaxation labeling processes,” in Proc. ESANN’95—3rd Europ. Symp. Artif. Neural Networks, Brussels, Belgium, 1995, pp. 223–228.

  57. G.A. Kohring, “Finite-state neural networks: A step toward the simulation of very large systems,” J. Stat. Phys., Vol. 62, pp. 563–576, 1991.

    Google Scholar 

  58. F. Crick and C. Asanuma, “Certain aspects of the anatomy and physiology of the cerebral cortex,” in Parallel Distributed Processing. Vol 2: Psychological and Biological Models, J.L. McClelland and D.E. Rumelhart (Eds.), MIT Press: Cambridge, MA, 1986, pp. 333–371.

    Google Scholar 

  59. J.A. Anderson, “Cognitive and psychological computation with neural models,” IEEE Trans. Syst. Man Cybern., Vol. 13, pp. 799–815, 1983.

    Google Scholar 

  60. J.A. Anderson and G.E. Hinton, “Models of information processing in the brain,” in Parallel Models of Associative Memory, G.E. Hinton and J.A. Anderson (Eds.), Erlbaum: Hillsdale, NJ, 1981, pp. 9–48.

    Google Scholar 

  61. D.J. Amit, Modeling Brain Function, Cambridge University Press: Cambridge, UK, 1989.

    Google Scholar 

  62. T. Hogg and B.A. Huberman, “Understanding biological computation: Reliable learning and recognition,” Proc. Natl. Acad. Sci. USA, Vol. 81, pp. 6871–6875, 1984.

    Google Scholar 

  63. M.A. Cohen and S. Grossberg, “Absolute stability of global pattern formation and parallel memory storage by competitive neural networks,” IEEE Trans. Syst. Man Cybern., Vol. 13, pp. 815–826, 1983.

    Google Scholar 

  64. G.E. Hinton and T.J. Sejnowski, “Learning and relearning in Boltzmann machines,” in Parallel Distributed Processing. Vol 1: Foundations, D.E. Rumelhart and J.L. McClelland (Eds.), MIT Press: Cambridge, MA, 1986, pp. 282–317.

    Google Scholar 

  65. C. Torras, “Relaxation and neural learning: Points of convergence and divergence,” J. Parallel Distrib. Computing, Vol. 6, pp. 217–244, 1989.

    Google Scholar 

  66. S.S. Yu and W.H. Tsai, “Relaxation by the Hopfield neural network,” Pattern Recognition, Vol. 25, pp. 197–209, 1992.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pelillo, M. The Dynamics of Nonlinear Relaxation Labeling Processes. Journal of Mathematical Imaging and Vision 7, 309–323 (1997). https://doi.org/10.1023/A:1008255111261

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008255111261

Navigation