When physics meets signal processing: Image and video denoising based on Ising theory
Introduction
Newton laws enable the prediction of the time evolution of a classical physical system given the initial conditions of its particles. However, when trying to describe macroscopic physical systems, with a typical number of 1023 particles, it is unpractical to solve all the equations of motion and find an analytical solution due to the computational burden involved. The best one can do is to find a statistical model describing the global characteristics of the system. This is why the tool-kit of statistical physics has become so important in modeling and solving problems from solid state and plasma physics to chemical and biological systems [1], [2], [3]. By modeling correctly the typical interaction between particles in the system, one can easily find the conditions for the equilibrium and its characteristic macroscopic parameters such as energy, temperature and pressure. In this paper, we discuss the similarities between these problems and the ones encountered when trying to extract as many information as possible regarding an image or a video signal.
In these cases as well, one tries to describe a large and complex system with specific correlations between its parts (pixels and/or frames). Additionally, in this field, there are cases where we have only partial knowledge about the pixels׳ values, hence there is a need to adopt probabilistic methods and estimations.
Utilizing this similarity, we employ our physical intuition regarding many-particles systems to the field of image processing and suggest a model for denoising of images and video signals degraded by additive external noise. Although the results are better than those of several known filters (Median, BM3D, Non-Local Means), we conclude that physical theory does not apply in a straightforward manner, as images do not always tend to minimal energy or maximal entropy similar to a physical system. Therefore, we suggest in this manuscript another improved model where Ising-like models are used together with basic image processing techniques, in order to achieve better image denoising.
The main contributions of this work are: employing for the first time an Ising-like model for denoising of colored images and video signals based on the L1 norm, utilizing Perona and Malik diffusion coefficient, histogram analysis and other methods for automatic choice of the model׳s parameters and in some cases achieving better results than other commonly used filters. We emphasize that while Bayesian and statistical physics-based restoration techniques for gray-scale image denoising were suggested in previous works (for example see, [4], [5], [6], [7], [8]), to the best of our knowledge, Ising-like models were not used before to restore colored images and video signals. Such a regime requires new tools and techniques to cope with the layered and multi-dimensional nature of the data. Particularly, the 3D Ising-like model and the L1 norm we used whose effectiveness has not been clarified yet in the research area of Markov random field. Furthermore, in order to improve past results, a better model for choosing the algorithm׳s parameters was needed.
The outline of the paper is as follows. In Section 2, we stress the analogy between physical systems and images and define the “thermodynamic” parameters characterizing an image. 3 The Ising model, 4 Ising-like models elaborate on Ising-like models and our proposed model respectively. In 5 Further adjustments, 6 Automatic parameters and noise estimation we focus on our recent algorithmic improvements where the latter section includes also automatic choice of parameters. Section 7 describes the theory and methods used to perform the simulation tasks given in this paper, while Section 8 presents the results. Section 9 summarizes the main contributions of the work and outlines future directions.
The paper is organized in a modular way, where in each section we try advance a step further towards efficient denoising of colored images.
Section snippets
Images as physical systems
Physical systems of many particles are described by average thermodynamic potentials [1]. The most basic of them is the internal energy, U, which can be represented in its differential form bywhere T is the temperature, S is the entropy, p describes the pressure and V is the volume. is the chemical potential of type i particles and is their number. E and H are the electric and magnetic fields, respectively. P is the polarization and M is the magnetization.
The Ising model
A well-known model in the field of solid-state physics is the Ising model [3], which assumes a simple two-body interaction in a lattice of many spin-1/2 particles. Spin 1/2 is an internal degree of freedom which can take one of the values 1 or −1.
The typical energy of the system in the one dimensional Ising model is defined as follows:where and denote the spins in the i-th and, j-th sites, describes the interaction between them, and is the external magnetic
Ising-like models
The original two dimensional Ising model enables considering up to 4 nearest neighbors to a central pixel. Also, pixels can have only two values. In the suggested novel scheme we use a somewhat improved model taking in account the effects of each pixel׳s 12 nearest neighbors (Fig. 1). In this way, one can increase the long-range correlation length. Furthermore, in order to eliminate unwanted artifacts, the weight of (the interaction strength) can be lowered as the neighbors are further away.
Further adjustments
Physics provides theoretical models and mathematical derivations. In order to utilize it correctly for image denoising, we had to implement several following adjustments:
- 1.
Adaptive weights for h and J: when considering their definitions, it is apparent that a high value of h is preferable when the reference image is known to be good, and high value of J is preferable in the “smooth” areas where the activity is low and neighboring pixels tend to be the same (see also the use of the Perona and
Automatic parameters and noise estimation
The next natural step is suggesting a fully automatic denoising. In order to do so, the strength of the external field was set to 1, and only the correlation strength and temperature were changed. Indeed, as can be deduced from Eqs. (4), (5), only the ration of J and h is important.
Secondly, the Perona and Malik model [13] was used (for the first time to our knowledge) in order to determine correlation strength within the Ising-like modelwhere g is the anisotropic diffusion
Metropolis algorithm and simulated annealing
The Metropolis algorithm [12] belongs to an important class of Monte-Carlo methods known as Markov Chain Monte-Carlo, which perform a sampling of the pdf׳s by constructing a Markov chain that has the desired distribution as its equilibrium distribution.
The algorithm is used for calculating equilibrium properties of two-dimensional substances which may be considered as composed of N classically interacting molecules separated by a minimal distance dij. Within the canonical ensemble [1], the
Results and comparisons
In this section, the results of our algorithm (4 Ising-like models, 5 Further adjustments, 6 Automatic parameters and noise estimation, 7 Metropolis algorithm and simulated annealing) are presented. The data was created as follows. For the original, data, we used several images and videos from the Matlab library. Then, two kinds of additive noise models were tested: Salt&Pepper and Gaussian noise. For Salt&Pepper, impulse noise was added independently to each color layer. For the Gaussian case,
Conclusions and future work
In this paper we have described an analogy between statistical physics and image processing, and demonstrated its usefulness, especially that of the Ising-like models. Developing a novel model based both on physical theory and practical considerations, we have demonstrated image and video denoising which excels the performance of the median filter by an average of 5.5 dB in the case of low impulse noise. When compared to BM3D or Wiener filter in the case of low Gaussian noise, the advantage of
References (22)
Thermal Physics
(1980)- et al.
Course of theoretical physics
(1959) Statistical Mechanics
(1972)- et al.
Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images.
IEEE Trans. Pattern Anal. Mach. Intell.
(1984) - et al.
Bayesian inference with wavelets: density estimation
J. Comput. Gr. Stat.
(1998) Statistical-mechanical approach to image processing
J. Phys. A
(2002)- et al.
A joint inter-and intrascale statistical model for Bayesian wavelet based image denoising
IEEE Trans. Image Process.
(2002) - M.C. Kandes, Statistical image restoration via the Ising model, in: Proceedings of the Final Project 2008....
- et al.
Optimization by simulated annealing
Science
(1983) - E. Cohen, R. Heiman, O. Hadar, Image and video restoration via Ising-like models, in: Proceedings of the IS&T/SPIE...
Cited by (5)
Image denoising searching similar blocks along edge directions
2017, Signal Processing: Image CommunicationCitation Excerpt :During the acquisition and transmission of an image, the contamination of the image with noise is unavoidable. Therefore, image denoising is a fundamental and widely studied problem in the field of image processing [1,2]. The goal of an image denoising algorithm is to efficiently remove noise while preventing blurring of edges.
Optimum wavelet based masking for the contrast enhancement of medical images using enhanced cuckoo search algorithm
2016, Computers in Biology and MedicineCitation Excerpt :Filtering is the process of selecting a desired band of signal. In image communication, filter based techniques are mainly used for restoration [16–18] and enhancement. Paul et al. [17] proposed a neighborhood filter for MR image enhancement.
Historical Document Image Denoising by Ising Model
2020, Proceedings - IEEE 18th International Conference on Dependable, Autonomic and Secure Computing, IEEE 18th International Conference on Pervasive Intelligence and Computing, IEEE 6th International Conference on Cloud and Big Data Computing and IEEE 5th Cyber Science and Technology Congress, DASC/PiCom/CBDCom/CyberSciTech 2020Removing phase transitions from gibbs measures
2017, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017Restoration of hot pixels in digital imagers using lossless approximation techniques
2015, Proceedings of SPIE - The International Society for Optical Engineering