When physics meets signal processing: Image and video denoising based on Ising theory

https://doi.org/10.1016/j.image.2015.02.007Get rights and content

Highlights

  • We present a detailed analogy between image processing and statistical physics.

  • We use a novel Ising-like model for denoising of damaged colored images and videos.

  • The algorithm chooses automatically the denoising parameters.

  • A novel simulated annealing code, based on the L1 norm is then run.

  • We denoise images and videos damaged by additive impulse or Gaussian noise.

  • The restoration results are better compared to other well-known filters.

Abstract

In this work we suggest a novel model for automatic noise estimation and image denoising. In particular, we investigate the useful affinity which arises between statistical mechanics and image processing, and describe a framework from which novel denoising algorithms can be derived: Ising-like models and simulated annealing techniques. This is the first time such algorithms are used for colored images and video denoising. Results, as well as benchmarks, suggest a significant gain in PSNR and SSIM in comparison to other filters, mainly in cases of low impulse noise. When hybridizing our models with other image processing techniques they are shown to be even more effective. Their major disadvantages- high complexity and limited applicability, are also discussed.

Introduction

Newton laws enable the prediction of the time evolution of a classical physical system given the initial conditions of its particles. However, when trying to describe macroscopic physical systems, with a typical number of 1023 particles, it is unpractical to solve all the equations of motion and find an analytical solution due to the computational burden involved. The best one can do is to find a statistical model describing the global characteristics of the system. This is why the tool-kit of statistical physics has become so important in modeling and solving problems from solid state and plasma physics to chemical and biological systems [1], [2], [3]. By modeling correctly the typical interaction between particles in the system, one can easily find the conditions for the equilibrium and its characteristic macroscopic parameters such as energy, temperature and pressure. In this paper, we discuss the similarities between these problems and the ones encountered when trying to extract as many information as possible regarding an image or a video signal.

In these cases as well, one tries to describe a large and complex system with specific correlations between its parts (pixels and/or frames). Additionally, in this field, there are cases where we have only partial knowledge about the pixels׳ values, hence there is a need to adopt probabilistic methods and estimations.

Utilizing this similarity, we employ our physical intuition regarding many-particles systems to the field of image processing and suggest a model for denoising of images and video signals degraded by additive external noise. Although the results are better than those of several known filters (Median, BM3D, Non-Local Means), we conclude that physical theory does not apply in a straightforward manner, as images do not always tend to minimal energy or maximal entropy similar to a physical system. Therefore, we suggest in this manuscript another improved model where Ising-like models are used together with basic image processing techniques, in order to achieve better image denoising.

The main contributions of this work are: employing for the first time an Ising-like model for denoising of colored images and video signals based on the L1 norm, utilizing Perona and Malik diffusion coefficient, histogram analysis and other methods for automatic choice of the model׳s parameters and in some cases achieving better results than other commonly used filters. We emphasize that while Bayesian and statistical physics-based restoration techniques for gray-scale image denoising were suggested in previous works (for example see, [4], [5], [6], [7], [8]), to the best of our knowledge, Ising-like models were not used before to restore colored images and video signals. Such a regime requires new tools and techniques to cope with the layered and multi-dimensional nature of the data. Particularly, the 3D Ising-like model and the L1 norm we used whose effectiveness has not been clarified yet in the research area of Markov random field. Furthermore, in order to improve past results, a better model for choosing the algorithm׳s parameters was needed.

The outline of the paper is as follows. In Section 2, we stress the analogy between physical systems and images and define the “thermodynamic” parameters characterizing an image. 3 The Ising model, 4 Ising-like models elaborate on Ising-like models and our proposed model respectively. In 5 Further adjustments, 6 Automatic parameters and noise estimation we focus on our recent algorithmic improvements where the latter section includes also automatic choice of parameters. Section 7 describes the theory and methods used to perform the simulation tasks given in this paper, while Section 8 presents the results. Section 9 summarizes the main contributions of the work and outlines future directions.

The paper is organized in a modular way, where in each section we try advance a step further towards efficient denoising of colored images.

Section snippets

Images as physical systems

Physical systems of many particles are described by average thermodynamic potentials [1]. The most basic of them is the internal energy, U, which can be represented in its differential form bydU=TdSpdV+i=idNi+EdP+HdMwhere T is the temperature, S is the entropy, p describes the pressure and V is the volume. μi is the chemical potential of type i particles and Ni is their number. E and H are the electric and magnetic fields, respectively. P is the polarization and M is the magnetization.

The Ising model

A well-known model in the field of solid-state physics is the Ising model [3], which assumes a simple two-body interaction in a lattice of many spin-1/2 particles. Spin 1/2 is an internal degree of freedom which can take one of the values 1 or −1.

The typical energy of the system in the one dimensional Ising model is defined as follows:E=12i.jJijSiSjihiSi.where Si and Sj denote the spins in the i-th and, j-th sites, Jijdescribes the interaction between them, and hi is the external magnetic

Ising-like models

The original two dimensional Ising model enables considering up to 4 nearest neighbors to a central pixel. Also, pixels can have only two values. In the suggested novel scheme we use a somewhat improved model taking in account the effects of each pixel׳s 12 nearest neighbors (Fig. 1). In this way, one can increase the long-range correlation length. Furthermore, in order to eliminate unwanted artifacts, the weight ofJ (the interaction strength) can be lowered as the neighbors are further away.

Further adjustments

Physics provides theoretical models and mathematical derivations. In order to utilize it correctly for image denoising, we had to implement several following adjustments:

  • 1.

    Adaptive weights for h and J: when considering their definitions, it is apparent that a high value of h is preferable when the reference image is known to be good, and high value of J is preferable in the “smooth” areas where the activity is low and neighboring pixels tend to be the same (see also the use of the Perona and

Automatic parameters and noise estimation

The next natural step is suggesting a fully automatic denoising. In order to do so, the strength of the external field was set to 1, and only the correlation strength and temperature were changed. Indeed, as can be deduced from Eqs. (4), (5), only the ration of J and h is important.

Secondly, the Perona and Malik model [13] was used (for the first time to our knowledge) in order to determine correlation strength within the Ising-like modelJ1g=1+|N|2,where g is the anisotropic diffusion

Metropolis algorithm and simulated annealing

The Metropolis algorithm [12] belongs to an important class of Monte-Carlo methods known as Markov Chain Monte-Carlo, which perform a sampling of the pdf׳s by constructing a Markov chain that has the desired distribution as its equilibrium distribution.

The algorithm is used for calculating equilibrium properties of two-dimensional substances which may be considered as composed of N classically interacting molecules separated by a minimal distance dij. Within the canonical ensemble [1], the

Results and comparisons

In this section, the results of our algorithm (4 Ising-like models, 5 Further adjustments, 6 Automatic parameters and noise estimation, 7 Metropolis algorithm and simulated annealing) are presented. The data was created as follows. For the original, data, we used several images and videos from the Matlab library. Then, two kinds of additive noise models were tested: Salt&Pepper and Gaussian noise. For Salt&Pepper, impulse noise was added independently to each color layer. For the Gaussian case,

Conclusions and future work

In this paper we have described an analogy between statistical physics and image processing, and demonstrated its usefulness, especially that of the Ising-like models. Developing a novel model based both on physical theory and practical considerations, we have demonstrated image and video denoising which excels the performance of the median filter by an average of 5.5 dB in the case of low impulse noise. When compared to BM3D or Wiener filter in the case of low Gaussian noise, the advantage of

References (22)

  • C. Kittel

    Thermal Physics

    (1980)
  • L.D. Landau et al.

    Course of theoretical physics

    (1959)
  • R.K. Pathria

    Statistical Mechanics

    (1972)
  • S. Geman et al.

    Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images.

    IEEE Trans. Pattern Anal. Mach. Intell.

    (1984)
  • P. Müller et al.

    Bayesian inference with wavelets: density estimation

    J. Comput. Gr. Stat.

    (1998)
  • K. Tanaka

    Statistical-mechanical approach to image processing

    J. Phys. A

    (2002)
  • A. Pizurica et al.

    A joint inter-and intrascale statistical model for Bayesian wavelet based image denoising

    IEEE Trans. Image Process.

    (2002)
  • M.C. Kandes, Statistical image restoration via the Ising model, in: Proceedings of the Final Project 2008....
  • S. Kirkpatrick et al.

    Optimization by simulated annealing

    Science

    (1983)
  • E. Cohen, R. Heiman, O. Hadar, Image and video restoration via Ising-like models, in: Proceedings of the IS&T/SPIE...
  • E. Cohen, M. Carmi,R. Heiman, O. Hadar, A. Cohen, Image restoration via Ising theory and automatic noise estimation,...
  • Cited by (5)

    • Image denoising searching similar blocks along edge directions

      2017, Signal Processing: Image Communication
      Citation Excerpt :

      During the acquisition and transmission of an image, the contamination of the image with noise is unavoidable. Therefore, image denoising is a fundamental and widely studied problem in the field of image processing [1,2]. The goal of an image denoising algorithm is to efficiently remove noise while preventing blurring of edges.

    • Optimum wavelet based masking for the contrast enhancement of medical images using enhanced cuckoo search algorithm

      2016, Computers in Biology and Medicine
      Citation Excerpt :

      Filtering is the process of selecting a desired band of signal. In image communication, filter based techniques are mainly used for restoration [16–18] and enhancement. Paul et al. [17] proposed a neighborhood filter for MR image enhancement.

    • Historical Document Image Denoising by Ising Model

      2020, Proceedings - IEEE 18th International Conference on Dependable, Autonomic and Secure Computing, IEEE 18th International Conference on Pervasive Intelligence and Computing, IEEE 6th International Conference on Cloud and Big Data Computing and IEEE 5th Cyber Science and Technology Congress, DASC/PiCom/CBDCom/CyberSciTech 2020
    • Removing phase transitions from gibbs measures

      2017, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017
    • Restoration of hot pixels in digital imagers using lossless approximation techniques

      2015, Proceedings of SPIE - The International Society for Optical Engineering
    View full text