Color image segmentation based on multi-level Tsallis–Havrda–Charvát entropy and 2D histogram using PSO algorithms

doi:10.1016/j.patcog.2019.03.011

Pattern Recognition

Volume 92, August 2019, Pages 107-118

https://doi.org/10.1016/j.patcog.2019.03.011 Get rights and content

Highlights

•
A generalized 2D multi-level thresholding criterion function is proved rigorously by mathematical induction method.
•
A multi-level thresholding scheme for a RGB color image is proposed.
•
PSO algorithm is applied to seek to optimal threshold values in a very reasonable computational time.
•
The segmented image is compared with the human segmentation from BSDS300 to evaluate the experiment results quantitatively and objectively.

Abstract

In this paper, we propose a multi-level thresholding model based on gray-level & local-average histogram (GLLA) and Tsallis–Havrda–Charvát entropy for RGB color image. We validate the multi-level thresholding formulation by using the mathematical induction method. We apply particle swarm optimization (PSO) algorithm to obtain the optimal threshold values for each component of a RGB image. By assigning the mean values from each thresholded class, we obtain three segmented component images independently. We conduct the experiments extensively on The Berkeley Segmentation Dataset and Benchmark (BSDS300) and calculate the average four performance indices (BDE, PRI, GCE and VOI) to show the effectiveness and reasonability of the proposed method.

Introduction

Image segmentation is a process in which an image is partitioned into non-overlapping regions such that each region is homogeneous and two arbitrary adjacent regions are heterogeneous. Thresholding method is one of the simplest and the most widely used segmentation techniques. Basically, there are two types of thresholding methods: bi-level and multi-level thresholding. Bi-level thresholding methods [1], [2], [3], [4], [5], [6], [7], [8] assume that the image has only two homogeneous regions: object and background. However, in many applications, one encounters images with multiple regions. In the past ten years, different kinds of multi-level thresholding methods and algorithms have been proposed [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19].

One-dimensional (1D) histogram has been used as a thresholding technique for years. It is derived from the gray level information of each pixel, but does not take into account the spatial correlation between a pixel and its neighbor pixels. In order to overcome this drawback, people presented several types of two-dimensional histograms (2D) in the past few years. The first thresholding model based on 2D histogram and Shannon entropy was introduced in 1989 [4]. This 2D histogram was constructed by using the gray level of each pixel and the average gray level from the local neighborhood of the pixel, and named as the gray-level & local-average histogram (GLLA). In the recent few years, several thresholding models by applying other types of 2D histograms, such as, gray-level & local-variance histogram (GLLV) [20], gray-level & local-entropy histogram (GLLE) [21], gray-level & spatial-correlation histogram (GLSC) [22], [23], gray-level & gradient-magnitude histogram (GLGM) [24] and 2D direction histogram (2DD) [25] and so on were developed. In this paper, we mainly apply GLLA histogram to build the proposed model. In general, thresholding methods based on 2D histogram [4], [6], [7], [11], [12], [17], [18], [19] perform better than the 1D histogram methods [3], [5], [8], [13], [14], [15], [16].

Recently, entropic thresholding techniques have attracted more and more attention [3], [4], [5], [6], [7], [8], [12], [13], [14], [15], [16], [17], [18], [19]. An entropic thresholding model is mostly about searching for the optimal threshold values by maximizing or minimizing a criterion function. In 2004 de Albuquerque et al. [5], a 1D bi-level thresholding model was presented based on the non-extensive property of the Tsallis entropy. In the same year, Sahoo and Arora [6] proposed a Rényi entropic bi-level thresholding method by using GLLA histogram. In 2006 [7], they combined the GLLA histogram with the Tsallis–Havrda–Charvát entropy, and proposed a Tsallis–Havrda–Charvát entropic bi-level thresholding model. As for the multi-level thresholding, formulating the multi-level criterion function based on 1D Shannon entropy [14] is not a difficult task because of the extensive property of Shannon entropy. But it is not easy to formulate the multi-level criterion function based on Tsallis entropy. Sparavigna [13], in 2015, formulated a multi-level thresholding model based on 1D Tsallis entropy. Furthermore, in 2017, Ishak [17], [18] presented two multi-level thresholding models based on GLLA histogram by using Rényi entropy and Tsallis entropy respectively. We point out that, in 2016, a multi-level thresholding model based on GLLA histogram and Kullback–Leibler divergence, instead of entropy, was developed in [11]. However, all the thresholding models mentioned above didnot deal with color image segmentation. In this paper, we extend the multi-level thresholding technique to RGB color image.

Generally speaking, in our real world, a color image provides better description of a scene than the gray-level image [1], [2], [3], [9], [10], [19], [26]. It is known that most of the segmentation methods for a gray-level image can be directly applied to each component of a RGB color image [26]. However, so far only a limited amount of the studies [10], [19] mentioned how to employ the multi-level thresholding techniques on a color image. The discussions above give us a motivation to come up with a RGB color image segmentation model under the framework of a multi-level thresholding technique by using Tsallis–Havrda–Charvát entropy and GLLA histogram.

It is pointed out that most of the bi-level thresholding methods can be extended to multi-level with some appropriate modifications. But, as for the 2D multi-level extension, it gives rise to the exponential increase of computational time [6], since exhaustively searching for the optimal threshold values of a multi-level thresholding is an NP-hard combinatorial optimization problem [17]. In order to reduce the computational time, in recent years, people have been focusing on the metaheuristic algorithms, such as, Artificial Bee Colony (ABC) approach [16], Differential Evolution (DE) approach [12], Quantum Genetic (QG) algorithm [17] and Particle Swarm Optimization (PSO) algorithm [11], [13], [14], [15], [27]. In this paper, we implement all the experiments by applying PSO algorithm because of its simplicity in concept, time efficiency and highly convergent properties.

We test our method on The Berkeley Segmentation Dataset and Benchmark (BSDS300) extensively by computing the average four performance indices (Probability Rand Index, PRI, Global Consistency Error, GCE, Variation of Information, VOI and Boundary Displacement Error, BDE). In this paper, we compare the average four performance indices from the proposed model with the results from [11], [13], [17] to illustrate the effectiveness and reasonability of our proposed model.

The main contributions of this paper are as follows:

(1)
We derive the generalized multi-level thresholding criterion function based on GLLA histogram and Tsallis–Havrda–Charvát entropy and validate this formulation rigorously by applying the mathematical induction method.
(2)
We propose a multi-level thresholding scheme for RGB color image which is the first attempt that has been done so far according to our research. We assign the mean values from each thresholded class to obtain three segmented component images independently. Then, we obtain a segmented RGB color image which is very close to the original image and has fewer color levels than the original image.
(3)
We demonstrate the dynamic idea of PSO algorithm by using a figure and employ the PSO algorithm to seek the optimal threshold values in a very reasonable computational time.
(4)
We compare the labeled segmented image with the benchmark images (ground truth, human segmentation) from BSDS300 to evaluate the proposed model quantitatively and objectively. We calculate the average four performance indices (PRI, GCE, VOI and BDE) of the proposed model and compare them with the results from [11], [13], [17].

This paper is organized as follows: In Section 2, we discuss the background knowledge about the GLLA histogram, Tsallis–Havrda–Charvát entropy and PSO method respectively. In Section 3, we formulate the multi-level thresholding criterion function for the proposed model and validate the formulation by using the mathematical induction method. Then we present a thresholding scheme for RGB color image. In Section 4, we report the effectiveness of our model by applying the PSO algorithm on BSDS300 and calculating the four performance indices. In Section 5, we present some concluding remarks and future work from our model.

Section snippets

Background materials

In this section, the details of the GLLA histogram, Tsallis–Havrda–Charvát entropy and PSO algorithm are discussed.

Assume that a gray-level image is a function f(x, y): $Z_{_{M}} \times Z_{_{N}} \to G,$ where $Z_{_{M}} = {1, 2, \dots, M}$ for M ≥ 2 and $G = {0, 1, \dots, 255}$ . Then a RGB color image is a vector function $\vec{f} (x, y) : Z_{M} \times Z_{N} : \to G \times G \times G$ such that: $[\vec{f} (x, y)] = [f_{r} (x, y), f_{g} (x, y), f_{b} (x, y)],$ where f_r(x, y), f_g(x, y), f_b(x, y) are red, green and blue components whose mixture generates any color that can be displayed. We use f_c(x, y) to represent an

The proposed model

In this section, we formulate the multi-level thresholding criterion function based on Tsallis–Havrda–Charvát entropy and GLLA histogram and justify this formulation by using the mathematical induction method. Then, we extend the model to each color component of a RGB image to obtain three thresholded components and a segmented color image.

Experiments

In this section, we discuss how to use PSO algorithm to search the optimal thresholds. Next we illustrate how to conduct all the experiments on the images from Berkeley Segmentation Dataset (BSDS300) and compare the average four performance indices (PRI, GCE, VOI and BDE) of our model with the result from 2D K-L divergence model [11], 1D Tsallis-based model [13] and 2D Rényi-based model [17].

Conclusion and future work

In this paper, we develop a multi-level RGB color image thresholding algorithm based on Tsallis–Havrda–Charvát entropy of degree α and GLLA histogram. We apply the PSO algorithm to obtain the better performance results from our proposed model compared with 1D Tsallis-based model [13], 2D K-L divergence model [11] and 2D Rényi-based model [17] (see the discussion from Section 4). Actually, the Tsallis–Havrda–Charvát entropy, $H_{n}^{α} (P)$ (see Eq. (3)), becomes Shannon entropy, S(p), when the degree α

Surina Borjigin graduated from the department of mathematics, University of Louisville, KY, USA with a Ph.D. in Applied & Industrial Mathematics in August 2018; M.A. in Mathematics, University of Louisville in May 2014; M.S. in Mathematics, Beihang University, Beijing, China in January 2012; B.S. in Information and Computer Science, Inner Mongolia University, Inner Mongolia, China in July 2007. Her research interest field is image segmentation and image thresholding.

References (38)

C.-K. Yang et al.
Reduction of color space dimensionality by moment - preserving thresholding and its application for edge detection in color images
Pattern Recognit. Lett.
(1996)
A.S. Abutaleb
Automatic thresholding of gray-level pictures using two-dimensional entropy
Comput. Vis. Graph. Image Process.
(1989)
P.K. Sahoo et al.
A thresholding method based on two dimensional renyi’s entropy
Pattern Recognit.
(2004)
P.K. Sahoo et al.
Image thresholding using two dimensional tsallis havrda charvát entropy
Pattern Recognit. Lett.
(2006)
X. Zhao et al.
A multilevel image thresholding segmentation algorithm based on two-dimensional k- l divergence and modified particle swarm optimization
Appl. Soft Comput.
(2016)
J. Chen et al.
Image thresholding segmentation based on two dimensional histogram using gray level and local entropy information
(2017)
Y. Xiao et al.
Entropic image thresholding based on glgm histogram
Pattern Recognit. Lett.
(2014)
A. Yimit et al.
2-D direction histogram based entropic thresholding
Neurocomputing
(2013)
H.D. Cheng et al.
Color image segmentation: advances and prospects
Pattern Recognit
(2001)
Z. Daróczy
Generalized information functions
Inf. Control
(1970)

A.Y. Yang et al.

Unsupervised segmentation of natural images via lossy data compression

Comput. Vision Image Understanding

(2008)

M. Celenk et al.

Optimal thresholding for color images

SPIE, Nonlinear Image Processing IX, San Jose, California, USA

(1998)

S. Sadek et al.

Entropic image segmentation: a fuzzy approach based on tsallis entropy

Int. J. Comput. Vis. Signal Process.

(2015)

M.P. de Albuquerque et al.

Image thresholding using tsallis entropy

Pattern Recognit. Lett.

(2004)

G.A. Azim et al.

A novel algorithm for image thresholding using non parametric fisher information

1st International Electronic Conference on Entropy and Its Application

(2014)

K.S. Tan et al.

Color image segmentation using histogram thresholding - fuzzy c-means hybrid approach

Pattern Recognit.

(2011)

E. Kurugollu et al.

Color image segmentation using histogram multithresholding and fusion

Image Vis. Comput.

(2001)

S. Sarkar et al.

Multilevel image thresholding based on 2d histogram and maximum tsalli entropy - a differential evolution approach

Trans. Image Process.

(2013)

A.C. Sparavigna

Tsallis entropy in bi-level and multi-level image thresholding

Int. J. Sci.

(2015)

Cited by (81)

IDRM: Brain tumor image segmentation with boosted RIME optimization
2023, Computers in Biology and Medicine
Timely diagnosis of medical conditions can significantly mitigate the risks they pose to human life. Consequently, there is an urgent demand for an effective auxiliary model that assists physicians in accurately diagnosing medical conditions based on imaging data. While multi-threshold image segmentation models have garnered considerable attention due to their simplicity and ease of implementation, the selection of threshold combinations greatly influences the segmentation performance. Traditional optimization algorithms often require substantial time to address multi-threshold image segmentation problems, and their segmentation accuracy is frequently unsatisfactory. As a result, metaheuristic algorithms have been employed in this domain. However, several algorithms suffer from drawbacks such as premature convergence and inadequate exploration of the solution space when it comes to threshold selection. For instance, the recently proposed optimization algorithm RIME, inspired by the physical phenomenon of rime-ice, falls short in terms of avoiding local optima and fully exploring the solution space. Therefore, this study introduces an enhanced version of RIME, called IDRM, which incorporates an interactive mechanism and Gaussian diffusion strategy. The interactive mechanism facilitates information exchange among agents, enabling them to evolve towards more promising directions and increasing the likelihood of discovering the optimal solution. Additionally, the Gaussian diffusion strategy enhances the agents' local exploration capabilities and expands their search within the solution space, effectively preventing them from becoming trapped in local optima. Experimental results on 30 benchmark test functions demonstrate that IDRM exhibits favorable optimization performance across various optimization functions, showcasing its robustness and convergence properties. Furthermore, the algorithm is applied to select threshold combinations for brain tumor image segmentation, and the results are evaluated using metrics such as Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM). The overall findings consistently highlight the exceptional performance of this approach, further validating the effectiveness of IDRM in addressing image segmentation problems.
Image segmentation of Intracerebral hemorrhage patients based on enhanced hunger Games search Optimizer
2023, Biomedical Signal Processing and Control
Medical diseases seriously affect human life and health, and a segmentation model that can effectively support doctors in making the correct diagnoses of medical disease images is needed. Multi-threshold image segmentation is famous for its simplicity and ease of implementation, but the choice of its threshold combination affects its performance, and traditional optimization algorithms fall into local optimality with significant time consumption when solving such problems. Therefore, metaheuristic algorithms have been applied to this field, but many have drawbacks, such as slow convergence, easy prematureness, and unbalanced performance when performing threshold selection. For instance, the Hunger Games Search (HGS) algorithm proposed last year is unsatisfactory regarding convergence accuracy and speed. Hence, an improved HGS (SCHGS) is proposed by combining the slime mould position update mechanism and chaotic optimal solution variation. The slime position update mechanism has a powerful exploration capability, which can help HGS increase the exploration of the search space and find the optimal solution as much as possible. On the other hand, the chaotic optimal solution variation strengthens the algorithm's local exploitation ability, which can effectively avoid falling into the local optimum. The experimental results on benchmark test functions indicate that the convergence performance of SCHGS is improved by 54% compared with the original algorithm, and there is a more obvious advantage in the convergence speed. In the application of threshold selection in brain hemorrhage image segmentation, the performance of the suggested method also improves by 0.08%, 0.55%, and 0.29% according to different evaluation metrics (FSIM, PSNR, and SSIM), further demonstrating the effectiveness of SCHGS in solving image segmentation problems.
Lévy flight and Chaos theory based metaheuristics for grayscale image thresholding
2023, Comprehensive Metaheuristics: Algorithms and Applications
The Levy flight and Chaos theory-based Gravitational Search Algorithm (LCGSA) has been applied for the image segmentation task. In LCGSA, the exploration is carried out by levy flight, while exploitation is guaranteed by chaotic maps. Besides, Kapur’s entropy method has been utilized to segment the sample image into various regions based on pixel concentration. Two famous benchmark images namely Cameraman and Lena have been considered for evaluating the segmentation capability of different LCGSA versions. Various performance metrics like standard deviation (SD), mean square error (MSE), structural similarity index measure (SSIM), feature similarity index measure (FSIM), etc. have been employed to validate the optimization performance of LCGSA. Moreover, the signed Wilcoxon rank-sum test has been applied to statistically verify the simulation results. Furthermore, eight state-of-the-art heuristic algorithms have been considered for comparative analysis. The MATLAB codes of the work will be publicly available on the author’s GitHub page, that is, https://github.com/SajadAHMAD1 and MathWorks website, https://www.mathworks.com/matlabcentral/profile/authors/6240015.
An efficient multi-threshold image segmentation for skin cancer using boosting whale optimizer
2022, Computers in Biology and Medicine
Citation Excerpt :
Traditional optimization methods have the disadvantages of falling into local optima and high computational complexity when solving this optimization task [61]. To solve these problems, many researchers have proposed the use of metaheuristic algorithms to help select the optimal threshold combinations, and have achieved remarkable results [62–64]. In this research paper, in order to be capable to solve the above problems effectively, we propose a 2D histogram based on NLM and gray value information, and combine the defined 2D Kapur's entropy and the proposed algorithm variants for the selection of thresholds to achieve a better segmentation.
Due to the terrible manifestations of skin cancer, it seriously disturbs the quality of life status and health of patients, so we needs treatment plans to detect it early and avoid it causing more harm to patients. Medical disease image threshold segmentation technique can well extract the region of interest and effectively assist in disease recognition. Moreover, in multi-threshold image segmentation, the selection of the threshold set determines the image segmentation quality. Among the common threshold selection methods, the selection based on metaheuristic algorithm has the advantages of simplicity, easy implementation and avoidable local optimization. However, different algorithms have different performances for different medical disease images. For example, the Whale Optimization Algorithm (WOA) does not give a satisfactory performance for thresholding skin cancer images. We propose an improved WOA (LCWOA) in which the Levy operator and chaotic random mutation strategy are introduced to enhance the ability of the algorithm to jump out of the local optimum and to explore the search space. Comparing with different existing WOA variants on the CEC2014 function set, our proposed and improved algorithm improves the efficiency of the search. Experimental results show that our method outperforms the extant WOA variants in terms of optimization performances, improving the convergence accuracy and velocity. The method is also applied to solve the threshold selection in the skin cancer image segmentation problem, and LCWOA also gives excellent performance in obtaining optimal segmentation results.
An efficient image segmentation method for skin cancer imaging using improved golden jackal optimization algorithm
2022, Computers in Biology and Medicine
Citation Excerpt :
In the real world, a color image generally provides a better depiction of a scene than a gray-level image. It is well-known that most gray-level image segmentation methods can be directly applied to each component of an RGB image, However, few studies [15–17] discuss how to use multilevel thresholding approaches on a color image . Borjigin et al. [15] focus on the most extensively used RGB color space for image segmentation.
Skin cancer is one of the worst cancers nowadays that poses a severe threat to the health and safety of individuals. Therefore, skin cancer classification and early diagnosis are recommended to preserve human life. Multilevel thresholding image segmentation is well-known and influential technique for extracting regions of interest from skin cancer images to improve the classification process. Therefore, this paper proposes an efficient version of the recently developed golden jackal optimization (GJO) algorithm, the opposition-based golden jackal optimizer (IGJO). The IGJO algorithm is used to solve the multilevel thresholding problem using Otsu’s method as an objective function. The proposed algorithm is compared with seven other meta-heuristic algorithms: whale optimization algorithm, seagull optimization algorithm, salp swarm algorithm, Harris hawks optimization, artificial gorilla troops optimizer, marine predators’ algorithms, and original GJO algorithm. The performance of the proposed algorithm is evaluated using four popular performance measures: peak signal-to-noise ratio, structure similarity index, feature similarity index, and mean square error. Experimental results show that the proposed algorithm outperforms other alternative algorithms in terms of PSNR, SSIM, FSIM, and MSE segmentation metrics and effectively resolves the segmentation problem.
Multilevel threshold image segmentation for COVID-19 chest radiography: A framework using horizontal and vertical multiverse optimization
2022, Computers in Biology and Medicine
Citation Excerpt :
Dhal et al. [25] presented a stochastic fractal search with a fuzzy entropy-based multilevel thresholding model for MTIS of color satellite images. Borjigin et al. [26] applied a particle swarm optimization algorithm to obtain the optimal threshold values for each component of an RGB image. Alwerfali et al. [27] developed an alternative MTIS method using a modified version of the salp swarm algorithm.
COVID-19 is currently raging worldwide, with more patients being diagnosed every day. It usually is diagnosed by examining pathological photographs of the patient's lungs. There is a lot of detailed and essential information on chest radiographs, but manual processing is not as efficient or accurate. As a result, how efficiently analyzing and processing chest radiography of COVID-19 patients is an important research direction to promote COVID-19 diagnosis. To improve the processing efficiency of COVID-19 chest films, a multilevel thresholding image segmentation (MTIS) method based on an enhanced multiverse optimizer (CCMVO) is proposed. CCMVO is improved from the original Multi-Verse Optimizer by introducing horizontal and vertical search mechanisms. It has a more assertive global search ability and can jump out of the local optimum in optimization. The CCMVO-based MTIS method can obtain higher quality segmentation results than HHO, SCA, and other forms and is less prone to stagnation during the segmentation process. To verify the performance of the proposed CCMVO algorithm, CCMVO is first compared with DE, MVO, and other algorithms by 30 benchmark functions; then, the proposed CCMVO is applied to image segmentation of COVID-19 chest radiography; finally, this paper verifies that the combination of MTIS and CCMVO is very successful with good segmentation results by using the Feature Similarity Index (FSIM), the Peak Signal to Noise Ratio (PSNR), and the Structural Similarity Index (SSIM). Therefore, this research can provide an effective segmentation method for a medical organization to process COVID-19 chest radiography and then help doctors diagnose coronavirus pneumonia (COVID-19).

View all citing articles on Scopus

Prasanna K. Sahoo who used to be a professor in the department of mathematics, University of Louisville. He passed away on 6/18/2017.

¹: Deceased June 18, 2017.

View full text

Color image segmentation based on multi-level Tsallis–Havrda–Charvát entropy and 2D histogram using PSO algorithms

Highlights

Abstract

Introduction

Section snippets

Background materials

The proposed model

Experiments

Conclusion and future work

Pattern Recognit. Lett.

Comput. Vis. Graph. Image Process.

Pattern Recognit.

Pattern Recognit. Lett.

Appl. Soft Comput.

Pattern Recognit. Lett.

Neurocomputing

Pattern Recognit

Inf. Control

Comput. Vision Image Understanding

Optimal thresholding for color images

SPIE, Nonlinear Image Processing IX, San Jose, California, USA

Entropic image segmentation: a fuzzy approach based on tsallis entropy

Int. J. Comput. Vis. Signal Process.

Image thresholding using tsallis entropy

Pattern Recognit. Lett.

A novel algorithm for image thresholding using non parametric fisher information

1st International Electronic Conference on Entropy and Its Application

Color image segmentation using histogram thresholding - fuzzy c-means hybrid approach

Pattern Recognit.

Color image segmentation using histogram multithresholding and fusion

Image Vis. Comput.

Multilevel image thresholding based on 2d histogram and maximum tsalli entropy - a differential evolution approach

Trans. Image Process.

Tsallis entropy in bi-level and multi-level image thresholding

Int. J. Sci.