Medical image segmentation using a contextual-constraint-based Hopfield neural cube

doi:10.1016/S0262-8856(01)00039-7

Image and Vision Computing

Volume 19, Issues 9–10, 1 August 2001, Pages 669-678

https://doi.org/10.1016/S0262-8856(01)00039-7 Get rights and content

Abstract

Neural-network-based image techniques such as the Hopfield neural networks have been proposed as an alternative approach for image segmentation and have demonstrated benefits over traditional algorithms. However, due to its architecture limitation, image segmentation using traditional Hopfield neural networks results in the same function as thresholding of image histograms. With this technique high-level contextual information cannot be incorporated into the segmentation procedure. As a result, although the traditional Hopfield neural network was capable of segmenting noiseless images, it lacks the capability of noise robustness. In this paper, an innovative Hopfield neural network, called contextual-constraint-based Hopfield neural cube (CCBHNC) is proposed for image segmentation. The CCBHNC uses a three-dimensional architecture with pixel classification implemented on its third dimension. With the three-dimensional architecture, the network is capable of taking into account each pixel's feature and its surrounding contextual information. Besides the network architecture, the CCBHNC also differs from the original Hopfield neural network in that a competitive winner-take-all mechanism is imposed in the evolution of the network. The winner-take-all mechanism adeptly precludes the necessity of determining the values for the weighting factors for the hard constraints in the energy function in maintaining feasible results. The proposed CCBHNC approach for image segmentation has been compared with two existing methods. The simulation results indicate that CCBHNC can produce more continuous, and smoother images in comparison with the other methods.

Introduction

Medical image segmentation serves as an important tool in clinical diagnosis and assessment. The result is useful to doctors for recognizing organs and tissues correctly, thus enhancing their diagnostic efficiency and minimizing their workload in medical image analysis. One challenge associated with image segmentation is the interference resulting from noise and artifacts. To eliminate the interference, incorporating contextual information is considered as one of the most effective approaches.

Another problem associated with image segmentation is that this work is usually extremely time-consuming. Recently, neural networks, with their features of fault tolerance and potential for parallel implementation, have been proposed as alternative approaches [1], [2], [3], [4], [5], [6], [7], [8]. Among them, Williams and Feng [8] embedded Bayesian classification theorem and conditional maximum likelihood into a multi-layer neural network for image segmentation. In their approach, eighteen to twenty-one features including color features, location and texture features, e.g. entropy, contrast and gray-level difference were fed to train the neural network via supervised learning. As we know, the supervised learning needs a set of targets to train the networks based on pre-defined image features. However, for different image modalities because of their different characteristics, the system requires different training images set as well as different image features. In addition, these kinds of neural network need a tedious training process resulting in complex computational time and training time.

On the other hand, the Hopfield neural networks, using unsupervised learning, adeptly precludes the necessity of pre-training the networks [3], [4], [5], [6], [7]. The segmentation using conventional Hopfield neural networks are formulated as a cost-function-minimization problem to perform gray level thresholding on the image histogram or the pixels' gray levels arranged in a one-dimensional array [3], [4], [5], [6], [7]. With this approach, the spatial relations of pixels are destroyed and thus, contextual information cannot be incorporated into the network's evolution.

In this paper, we propose a three-dimensional Hopfield neural network, called contextual-constraint based Hopfield neural cube (CCBHNC), for medical image segmentation. Unlike other neural networks [3], [4], [5], [6], [7], CCBHNC uses a three-dimensional architecture with pixel classification implemented on its third dimension. With the three-dimensional architecture, the network is capable of incorporating each pixel's contextual information into a pixel classification, achieving a high-level vision model approach. Consequently, the effect of minor details or noise can be effectively removed, and the drawback of disconnected fractions can be further overcome.

For optimization, constraints usually play a very important role in solving the problems. There are two types of constraints: soft constraints and hard constraints. Soft constraints are used to help the network to obtain more desirable results [9], [10]. It is not necessary to satisfy all soft constraints as long as a proportional balance is retained among them. On the contrary, hard constraints enforce conditions that enable the network to reach a feasible solution. Therefore, the hard constraints must be absolutely satisfied. In the past, some hard constraints had to be added to the energy function of the Hopfield networks for it to reach a reasonable solution. However, it has proved to be very difficult to determine the weighting factors between hard constraints and the problem-dependent energy function. Improper parameters would lead to unfeasible solutions. Recently, Chung et al. [11] proposed a concept of competitive learning to exclude the hard constraints in the Hopfield network and eliminate the issue of determination of weighting factors. This proposed competitive learning rule is also adopted in CCBHNC.

In the proposed CCBHNC, two soft constraints are also introduced in the course of image segmentation. One gives the restriction in which the gray levels assigned to the same class have the minimum Euclidean distance measure. The other uses contextual information to force neighboring pixels to be classified into the same class. Given the two soft constraints, CCBHNC takes into consideration both the global gray level distribution and contextual information of pixels to obtain desirable segmentation results from noisy images.

To validate its effectiveness, the CCBHNC has been tested on various kinds of medical images including CT, MRI and SPECT. The simulation results show that CCBHNC can produce more continuous and smoother images in comparison with the dynamic thresholding [13] and CHNN [6] methods. Furthermore, the adoption of the competitive learning rule in CCBHNC relieves us of the burden of determining the proper values for the weighting factors and facilitates fast converge of the network.

The remainder of this paper is organized as follows. In Section 2, the architecture and algorithm of CCBHNC is proposed. In Section 3, mathematical derivations for the convergence of the CCBHNC are given. In Section 4, a comparative study between the proposed method and two existing methods is conducted. Finally, conclusions are drawn in Section 5.

Section snippets

A contextual-constraint-based Hopfield neural cube

In general, high-level image segmentation can be considered as a clustering process, simultaneously considering the target pixel feature and its contextual information. This high-level image segmentation can be achieved through minimizing an objective function, which is composed of the global gray level and contextual information. The objective function of this purpose can be represented as follows: $J= ∑ x=1 N ∑ i=1 N ∑ T=1 k ∑ y=1 N ∑ j=1 N C_{x,i} =C_{y,j} =T |g_{x,i} −g_{y,j} | Max (G) + ∑ y N ∑ j N y,j∈Ω_{x,i}^{r} f(C_{x,i},C_{y,j}),$ where k is

The convergence of the CCBHNC

In what follows, we will prove that the energy function of the proposed CCBHNC always decreases during network evolution. This implies that the network will converge to a stable state. Consider the energy function of the CCBHNC: $E= 12 ∑ k=1 C ∑ z=1 C ∑ x=1 N ∑ y=1 N (y,j)≠(x,i) ∑ i=1 N ∑ j=1 N [Ad_{x,i;y,j} δ_{k,z} +BΦ_{x,i}^{p,q} (y,j)(1−δ_{k,z})]V_{x,i,k} V_{y,j,z} .$ According to the architecture of CCBHNC, only the outputs of the neurons with the same height and the outputs of the neighboring neurons with different heights may affect the

Experimental results

To show that the proposed CCBHNC has the capability of image segmentation and robustness to noise, four cases of different modality medical images, including computer generated phantom image (Fig. 2(a)), skull-based CT (Fig. 5(a)), abdominal MRI (Fig. 6(a)), and single photon emission computed tomography (SPECT), Fig. 7(a) were tested. All the cases used for evaluating the CCBHNC were collected from the National Cheng Kung University Hospital. The MRI images were taken from the Siemen's

Conclusions

Although image processing and computer vision techniques have been developed for about 30 years, and are commonly applied for medical image processing, it is well known that their capabilities are far inferior to human vision systems. One of the main reasons is that these techniques only observe an image of individual pixel's gray value, while human vision systems observe an image from a global high-level perspective where contextual information is simultaneously included.

In this paper, we

Acknowledgements

The author wishes to thank Dr Jzau-Sheng Lin for kindly providing the image data, M.D. Ping-Hong Lai and Horng-Ming Tsai for evaluating the segmentation results, which greatly helped to improve the quality of the manuscript. This work was partly supported by NSC and Ministry of Education, Academic Excellence Grant 89-B-FA08-1-4.

References (14)

S. Haring et al.
Kohonen networks for multiscale image segmentation
Image Vision Comput.
(1994)
R. Sammouda et al.
Hopfield neural network for the multichannel segmentation of magnetic resonance cerebral images
Pattern Recogn.
(1997)
P.C. Chung et al.
Polygonal approximation using a competitive Hopfield neural network
Pattern Recogn.
(1994)
W.C. Lin et al.
Constraint satisfaction neural networks for image segmentation
Pattern Recogn.
(1992)
S.C. Armatur et al.
Optimization neural networks for the segmentation of magnetic resonance images
IEEE Trans. Med. Imag.
(1992)
R. Sammouda et al.
A Comparison of Hopfield neural network and Boltzmann machine in segmentation MR images of brain
IEEE Trans. Nucl. Sci.
(1996)
K.S. Cheng et al.
The application of competitive Hopfield neural network to medical image segmentation
IEEE Trans. Med. Imag.
(1996)

There are more references available in the full text version of this article.

Cited by (33)

Decision Support Tools, Systems, and Artificial Intelligence in Cardiac Imaging
2018, Canadian Journal of Cardiology
Noninvasive cardiac imaging is widely used for the diagnosis and management of cardiac patients. The increasing demand for cardiac imaging begins to exceed the number of available interpreting physicians, leaving less time to interpret studies. In addition, the busy clinician is facing the increasingly daunting task of keeping abreast of current medical advancements and the ongoing changes in disease diagnosis and therapy. Committing to memory and recalling such large volumes of information is challenging and is responsible for difficulties in adopting the rapid changes in imaging practice, and is likely partially responsible for errors in patient diagnosis and management. Diagnostic errors rank high in the cause of death in the United States, and are more common than any other medical error and are responsible for most malpractice claims. Most of these errors are related to cognitive errors. The use of artificial intelligence systems that can serve as complementary methods to assist humans with decision making can potentially prevent these errors. The past decades witnessed the development and integration of these tools, which can assist physicians with image interpretation. These tools work to optimize image quality for better visualization and accompany all imaging modalities, starting from patient selection for the appropriate test, patient preparation, image acquisition, processing, and finally interpretation. Current and future directions for technologies that support cardiac imaging physicians are discussed in this review.
L’imagerie cardiaque non invasive est couramment utilisée pour le diagnostic et la prise en charge des patients atteints d’une cardiopathie. La demande croissante en services d’imagerie cardiaque commence à dépasser le nombre de médecins disponibles pour l’interprétation de ces examens, ce qui laisse à ces derniers peu de temps à consacrer à des études sur l’interprétation. De surcroît, le clinicien débordé doit aussi s’acquitter de la lourde obligation de se tenir informé des avancées médicales et des changements les plus récents en matière de diagnostics et de traitements. La mémorisation et le rappel de ces importantes quantités d’information constituent un défi de taille qui explique les difficultés des médecins à adopter les changements rapides de pratiques en matière d’imagerie et pourrait expliquer en partie les erreurs de diagnostic et de prise en charge des patients. Les erreurs de diagnostic figurent parmi les premières causes de décès aux États-Unis, sont plus fréquentes que toute autre erreur médicale et sont à l’origine de la majorité des actions en responsabilité médicale. La plupart de ces erreurs ont une composante cognitive. Le recours à des systèmes d’intelligence artificielle qui pourraient servir de méthodes complémentaires pour aider la prise de décisions par les humains pourrait potentiellement prévenir ces erreurs. Ces dernières décennies ont vu la mise au point et l’intégration de tels outils, qui peuvent aider les médecins à interpréter les images. Ces outils ont pour fonction d’optimiser la qualité de l’image pour faciliter sa visualisation et d’accompagner toutes les modalités de l’imagerie, de la sélection de l’examen approprié pour le patient à l’interprétation finale, en passant par la préparation du patient, l’acquisition et le traitement de l’image. Dans cet article, nous passons également en revue les orientations actuelles et futures des technologies destinées à seconder les médecins spécialisés en imagerie cardiaque.
Segmentation of abdominal organs from CT using a multi-level, hierarchical neural network strategy
2014, Computer Methods and Programs in Biomedicine
Citation Excerpt :
An alternative method is to use Hopfield NN with unsupervised learning, where the segmentation procedure is formulated as a cost minimization problem [32]. For improving traditional design of Hopfield networks (i.e. architectural limitations), contextual information is incorporated in more advanced designs [9,36]. However, they could only make partial improvements over the traditional ones, since the results show that they fail at the regions where the gray level of the desired region is too close to the adjacent tissues.
Precise measurements on abdominal organs are vital prior to the important clinical procedures. Such measurements require accurate segmentation of these organs, which is a very challenging task due to countless anatomical variations and technical difficulties. Although, several features with various classifiers have been designed to overcome these challenges, abdominal organ segmentation via classification is still an emerging field in order to reach desired precision. Recent studies on multiple feature–classifier combinations show that hierarchical systems outperform composite feature–single classifier models. In this study, how hierarchical formations can translate to improved accuracy, when large size feature spaces are involved, is explored for the problem of abdominal organ segmentation. As a result, a semi-automatic, slice-by-slice segmentation method is developed using a novel multi-level and hierarchical neural network (MHNN). MHNN is designed to collect complementary information about organs at each level of the hierarchy via different feature–classifier combinations. Moreover, each level of MHNN receives residual data from the previous level. The residual data is constructed to preserve zero false positive error until the last level of the hierarchy, where only most challenging samples remain. The algorithm mimics analysis behaviour of a radiologist by using the slice-by-slice iteration, which is supported with adjacent slice similarity features. This enables adaptive determination of system parameters and turns into the advantage of online training, which is done in parallel to the segmentation process. Proposed design can perform robust and accurate segmentation of abdominal organs as validated by using diverse data sets with various challenges.
An efficient neural network based method for medical image segmentation
2014, Computers in Biology and Medicine
Citation Excerpt :
Artificial neural network (ANN) has been widely used in medical image analysis fields such as segmentation, data compression, image enhancement and noise suppression [11,12]. Multilayer perceptron (MLP), self-organizing maps (SOM), Hopfield and pulse coupled neural networks have been also utilized for medical image segmentation [13–18,29,30]. SOM network is one of the most suitable networks used for segmentation.
The aim of this research is to propose a new neural network based method for medical image segmentation. Firstly, a modified self-organizing map (SOM) network, named moving average SOM (MA-SOM), is utilized to segment medical images. After the initial segmentation stage, a merging process is designed to connect the objects of a joint cluster together. A two-dimensional (2D) discrete wavelet transform (DWT) is used to build the input feature space of the network. The experimental results show that MA-SOM is robust to noise and it determines the input image pattern properly. The segmentation results of breast ultrasound images (BUS) demonstrate that there is a significant correlation between the tumor region selected by a physician and the tumor region segmented by our proposed method. In addition, the proposed method segments X-ray computerized tomography (CT) and magnetic resonance (MR) head images much better than the incremental supervised neural network (ISNN) and SOM-based methods.
Medical image analysis with artificial neural networks
2010, Computerized Medical Imaging and Graphics
Citation Excerpt :
Furthermore, neural networks have the capability of optimising the relationship between the inputs and outputs via distributed computing, training, and processing, leading to reliable solutions desired by specifications, and medical diagnosis often relies on visual inspection, and medical imaging provides the most important tool for facilitating such inspection and visualisation. Medical image segmentation and edge detection remains a common problem and foundational for all medical imaging applications [15–25]. Any content analysis and regional inspection requires segmentation of featured areas, which can be implemented via edge detection and other techniques.
Given that neural networks have been widely reported in the research community of medical imaging, we provide a focused literature survey on recent neural network developments in computer-aided diagnosis, medical image segmentation and edge detection towards visual content analysis, and medical image registration for its pre-processing and post-processing, with the aims of increasing awareness of how neural networks can be applied to these areas and to provide a foundation for further research and practical development. Representative techniques and algorithms are explained in detail to provide inspiring examples illustrating: (i) how a known neural network with fixed structure and training procedure could be applied to resolve a medical imaging problem; (ii) how medical images could be analysed, processed, and characterised by neural networks; and (iii) how neural networks could be expanded further to resolve problems relevant to medical imaging. In the concluding section, a highlight of comparisons among many neural network applications is included to provide a global view on computational intelligence with neural networks in medical imaging.
Neuro semantic thresholding using OCR software for high precision OCR applications
2010, Image and Vision Computing
Citation Excerpt :
The images are captured by the machine’s camera and transferred to a personal computer for testing. The correctness of the algorithm has been tested binarizing the votes with different algorithms and reading the results [19,20], character by character, using a commercial OCR [42]. The images of each set are depicted in Figs. 7 and 8.
This paper describes a novel approach to binarization techniques. It presents a way of obtaining a threshold that depends both on the image and the final application using a semantic description of the histogram and a neural network. The intended applications of this technique are high precision OCR algorithms over a limited number of document types.
The input image histogram is smoothed and its derivative is found. Using a polygonal version of the derivative and the smoothed histogram, a new description of the histogram is calculated. Using this description and a training set, a general neural network is capable of obtaining an optimum threshold for our application.
A hierarchical evolutionary algorithm for automatic medical image segmentation
2009, Expert Systems with Applications
Image segmentation denotes a process of partitioning an image into distinct regions. A large variety of different segmentation approaches for images have been developed. Among them, the clustering methods have been extensively investigated and used. In this paper, a clustering based approach using a hierarchical evolutionary algorithm (HEA) is proposed for medical image segmentation. The HEA can be viewed as a variant of conventional genetic algorithms. By means of a hierarchical structure in the chromosome, the proposed approach can automatically classify the image into appropriate classes and avoid the difficulty of searching for the proper number of classes. The experimental results indicate that the proposed approach can produce more continuous and smoother segmentation results in comparison with four existing methods, competitive Hopfield neural networks (CHNN), dynamic thresholding, k-means, and fuzzy c-means methods.

View all citing articles on Scopus

View full text

Medical image segmentation using a contextual-constraint-based Hopfield neural cube

Abstract

Introduction

Section snippets

A contextual-constraint-based Hopfield neural cube

The convergence of the CCBHNC

Experimental results

Conclusions

Acknowledgements

Image Vision Comput.

Pattern Recogn.

Pattern Recogn.

Constraint satisfaction neural networks for image segmentation

Pattern Recogn.

Optimization neural networks for the segmentation of magnetic resonance images

IEEE Trans. Med. Imag.

A Comparison of Hopfield neural network and Boltzmann machine in segmentation MR images of brain

IEEE Trans. Nucl. Sci.

The application of competitive Hopfield neural network to medical image segmentation

IEEE Trans. Med. Imag.