A Novel Framework for Segmentation of Secretory Granules in Electron Micrographs
Graphical abstract
Introduction
Biomedical image processing helps researchers to extract information from images they need to analyze. As microscopes become more sophisticated and are able to achieve better resolution, researchers are able to observe cellular processes in more detail. With the increased viewing power, new image processing algorithms need to be developed in order to extract as much information from the images as possible. Currently, scientists mainly resort to slow manual analysis to extract and analyze information from limited amounts of image data. Image informatics has become the rate-limiting factor in realizing the full potential of dynamic cellular and molecular imaging studies (Zhou and Wong, 2006).
Insulin is the only hormone in mammals able to lower blood glucose levels and consequently it is vital for life (Göpel et al., 2004, Straub et al., 2004). Disbalance of insulin levels may result in diabetes. The hormone is secreted into the blood stream by beta cells inside islets of Langerhans of the pancreas. After its biosynthesis, insulin is processed and packaged in granules, surrounded by a membrane. In this work, transmission electron microscopy (TEM) images of beta cells have been acquired to examine the physical dimensions of insulin granules. Typically, insulin granules are described as organelles containing a dense core, surrounded by a halo and an enclosing membrane (Fig. 1). Along with the granules, other structures and organelles are present in the images, such as mitochondria, which can in some orientations be mistaken for granules. Granule cores are typically round and can vary in intensity and texture. Some granules do not have a surrounding halo, depending on the fixation procedure. Also, in some cases granule halos only have a partially visible membrane. All of these factors make accurately segmenting granules a challenging task.
Biologists mainly use open source software, such as ImageJ (Walter et al., 2010), MetaMorph or CellProfiler (Jones et al., 2008) to perform image processing tasks. A common method for microscopy image segmentation is image thresholding. Fiji—which is based on ImageJ—includes many thresholding methods, for example in Otsu (1979) and mean thresholding. A level-set active contour method is also available for use. It is similar to the gradient based active contour presented in Caselles et al. (1995). In order to achieve the desired segmentation, the biologist will have to employ multiple procedures and, in some environments, create a program flow. This requires prior image processing knowledge on the side of the biologist. While suitable for simple image processing tasks such analysis tools are too general in their scope and do not provide a solution to the challenging task of granule detection in TEM images. A number of approaches for microscopy cell segmentation have been previously described. Methods based on watershed transforms (Nguyen and Ji, 2008), multiscale products (Olivo-Marin, 2002), the Hough transform (Guan and Yan, 2011, Liu et al., 2006) and the sliding band filter (Quelhas et al., 2010) have been proposed, however as we will also show later these methods are not readily applicable to segmenting TEM images of beta cell granules. Horváth et al. (2009) proposed a shape prior for tree crown extraction, based on higher order active contours. This method incorporates specific shape information into the model, (the tree crown radius). This however would not be suited to granule segmentation, due to the variability in granule sizes. Recently, a method was developed to analyze insulin granules in TEM images (Fava et al., 2012). It was developed for a specific (expensive) software package: Developer XD by Definiens. The method described in this paper has a lower miss rate (4%) than that of the method used by Fava et al. (2012) (17%).2
Active contours are a popular method used in segmentation (Volkmann, 2010). Techniques using active contours have been developed for microscopic image segmentation, (Dzyubachyk et al., 2010, Zimmer et al., 2002, Dufour et al., 2005). We propose a segmentation algorithm that is based on the level-set active contour model specifically targeted at granule core segmentation of beta cell TEM images. Being able to split and merge easily allows the level-set method to segment objects with unknown topology, which is the case in granule segmentation (Angelini et al., 2005). Our granule segmentation algorithm is broken down into two major sections: core segmentation and membrane (halo) segmentation. We initially present preliminary results that can be found in (Nam et al., 2012). Two novel level-set active contour segmentation energy functionals are presented and employed for core segmentation. The first is based on intensity information and so can detect objects with a weak border. A novel shape regularizer is included in the energy functional to promote granule segmentation. This rewards the contour for having a low perimeter to area ratio. Without the shape regularizer, granules are oversegmented while unwanted details in the cytoplasm are also segmented. This method provides a good starting point for core segmentation but needs refining.
The second contribution is a dual level-set active contour based on the energy functional presented in Li et al. (2008). The dual active contour method is used to prevent the contour from getting stuck in a local minimum, therefore guiding the contour to the best solution. This energy functional is able to segment inhomogeneous images. The dual active contour is applied on each granule core to get the best possible segmentation. Once cores are segmented, obtaining their area is trivial.
The second section of the granule segmentation algorithm is halo extraction. This section uses as inputs the TEM image segmented by our first level-set method, the segmented cores and the TEM image. Membrane segmentation is done on a single granule basis. Using the center of a granule core as a starting point, the core is removed and the membrane is sampled from the image segmented with our level-set method. We sample from the image segmented by our first level-set method because the membrane is present in this image, but it is over segmented. The sampled points are then bridged and scaled, before a vector field convolution (VFC) snake (Li and Acton, 2007) is applied in order to fill any gaps in the membrane.
Granules do not always possess a surrounding halo (Fava et al., 2012). In order to check if our segmentation is correct, the segmented membrane is overlapped with the image segmented by our first level-set method. If there is no significant overlap, the granule was deemed not to have a halo. To compliment this, a modified sliding band filter (SBF) is applied on the TEM image. The SBF is part of the family of convergence filters and is suited to detect convex objects (Quelhas et al., 2010). The SBF is based on gradient convergence and not on intensity. If a granule does have a halo that is properly segmented, the gradient directions around the membrane will converge to the same point within the center of the granule. Granule membranes often contain weak edges, which is another reason that justifies the use of the SBF. We present a detection criterion to evaluate the membrane convergence and exclude membranes that do not meet the minimum convergence level. The remainder of the paper is organized as follows: Section 2 describes our level-set regularizing term to reduce cytoplasmic segmentation and our dual level-set method for accurately segmenting single granule cores. Section 3 describes our algorithm for granule core segmentation. We present our methods for granule membrane segmentation and verification in Section 4. In Section 5 we validate our approach and conclude in Section 6.
Section snippets
Geometric active contours
It can be seen from Fig. 1 that TEM images of beta cells in the islet of Langerhans are complex and contain many interacting structures. Due to the complexity of the image, the core segmentation process is broken down into three steps. The first step utilizes our locally scalable region-based level-set active contour, with a shape regularizer, to provide a good pre-segmentation of the cores. Other organelles and cell structures are segmented in the first step; the second step uses morphological
Granule core segmentation
Granule core segmentation is performed in three stages. The input for this fully automated algorithm is a beta cell TEM image. The proposed region-based active contour discussed in Section 2.3 is first applied to the image. In order to initialize the active contour, a mask is obtained from the image . We use a windowed mean thresholding method (Davies, 2005). This is computed quickly and gives a good starting point for the active contour. A square window of size pixels is used.
Segmentation of granule membranes
Some granules have a surrounding halo, as seen in Fig. 6. The distribution of granules with and without halos is influenced by the fixation procedure, hence to have accurate dimensions for granules, segmentation of the halo is necessary. The TEM image, along with the core segmentations and an image of the TEM segmented by (11) (BW) are used as inputs to the algorithm. The membrane segmentation algorithm proceeds through the following steps: morphological operations, sampling, VFC snake and
Experimental results
Our algorithm is initially applied to seven images, four fixed chemically and three fixed using cryofixation. All images are pixels but are from an original format of a 3∗3 tiled series containing at least one complete beta cell and a number (>5) of partially visible surrounding beta cells for each image. Each image is divided into 16 smaller images, for easier computation, giving 112, pixel images. These give about 1700 granules, to be processed and compared to our ground
Conclusion
In this paper we present an automatic segmentation approach, for insulin granules in TEM images of beta cells in the rat islet of Langerhans. Our method is designed to segment granule cores as well as the surrounding halos. Core segmentation is separated into three steps: pre-segmentation using our novel level-set shape regularizer, morphological cleaning and an accurate final segmentation using our novel localized dual level-set. Our level-set contributions are based on region-based active
References (38)
- et al.
A higher-order active contour model of a ‘gas of circles’ and its application to tree crown extraction
Pattern Recognit.
(2009) Extraction of spots in biological images using multiscale products
Pattern Recogn.
(2002)- et al.
Fronts propagating with curvature dependent speed: algorithms based on Hamilton–Jacobi formulations
J. Comput. Phys.
(1988) - et al.
Radial symmetries based decomposition of cell clusters in binary and gray level images
Pattern Recogn.
(2008) Chapter two – methods for segmentation and interpretation of electron tomographic reconstructions
Cryo-EM, Part C: Anal. Interpret. Case Stud.
(2010)- et al.
State of the art of level set methods in segmentation and registration of medical imaging modalities
Handbook Biomed. Image Anal. Registr. Models
(2005) Random forests
Mach. Learn.
(2001)- et al.
Geodesic active contours
Int. J. Comput. Vis.
(1995) - et al.
Active contours without edges
IEEE Trans. Image Process.
(2001) Machine Vision, Third Edition: Theory, Algorithms, Practicalities (Signal Processing and its Applications)
(2005)
Use of the Hough transformation to detect lines and curves in pictures
Commun. ACM
Segmenting and tracking fluorescent cells in dynamic 3-d microscopy with coupled active surfaces
IEEE Trans. Image Process.
Advanced level-set-based cell tracking in time-lapse fluorescence microscopy
IEEE Trans. Med. Imag.
Novel standards in the measurement of rat insulin granules combining electron microscopy, high-content image analysis and in silico modelling
Diabetologia
Capacitance measurements of exocytosis in mouse pancreatic alpha-, beta- and delta-cells within intact islets of Langerhans
J. Physiol.
Blood cell image segmentation based on the Hough transform and fuzzy curve tracing
Proc. ICMLC
CellProfiler analyst: data exploration and analysis software for complex image-based screens
BMC Bioinform.
Snakes: active contour models
Int. J. Comput. Vis.
Finding circles by an array of accumulators
Commun. ACM
Cited by (0)
- 1
Shared last author.