Elsevier

Neurocomputing

Volume 229, 15 March 2017, Pages 23-33
Neurocomputing

Abdominal adipose tissues extraction using multi-scale deep neural network

https://doi.org/10.1016/j.neucom.2016.07.059Get rights and content

Abstract

Segmentation of abdominal adipose tissues (AAT) into subcutaneous adipose tissues (SAT) and visceral adipose tissues (VAT) is of crucial interest for managing the obesity. Previous methods with raw or hand-crafted features rarely work well on large-scale subject cohorts, because of the inhomogeneous image intensities, artifacts and the diverse distributions of VAT. In this paper, we propose a novel two-stage coarse-to-fine algorithm for AAT segmentation. In the first stage, we formulate the AAT segmentation task as a pixel-wise classification problem. First, three types of features, intensity, spatial and contextual features, are extracted. Second, a new type of deep neural network, named multi-scale deep neural network (MSDNN), is provided to extract high-level features. In the second stage, to improve the segmentation accuracy, we refine coarse segmentation results by determining the internal boundaries of SAT based on coarse segmentation results and the continuous of SAT internal boundaries. Finally, we demonstrate the efficacy of our algorithm for both 2D and 3D cases on a wide population range. Compared with other algorithms, our method is not only more suitable for large-scale dataset, but also achieves better segmentation results. Furthermore, our system takes about 2 s to segment an abdominal image, which implies potential clinical applications.

Introduction

Obesity has become a worldwide healthcare issue. Statistic data from World Health Organization (WHO) showed that about 39% adults were overweight, with 13% of them obese in 2014. Obesity has been identified as a crucial risk factor for metabolic and cardiovascular diseases, such as, type 2 diabetes mellitus, atherosclerosis and hypertension [1], [2]. Moreover, obesity can accelerate epigenetic aging of human liver [3], and even induce gut microbial metabolite which promotes liver cancer [4]. Substantial researches [5] have shown that the accumulation of visceral adipose tissue (VAT), which covers internal organs within the abdomen, has a stronger association with obesity-related illnesses, compared to subcutaneous adipose tissue (SAT). Therefore, accurate and rapid measurement of VAT and SAT will be beneficial to assess the severity of the obesity-related diseases.

Currently, measuring the volumes of VAT are mainly based on segmentation of abdominal MR images, which is a challenging task due to the inhomogeneous intensities, low intensity contrast, large shape variation of VAT and complex structures of VAT and SAT (see Fig. 1). Manual and automatic segmentation methods are two mainstream ways for abdominal adipose tissues (AAT) segmentation. Manual segmentation is accurate, but usually laborious, subjective, and time consuming. Fully-automatic segmentation methods [6], [7], [8], [9], [10], [11] are more attractive. However, most existing automatic segmentation algorithms of AAT use hand-crafted features, such as intensity and shapes of different tissues, which do not explore and exploit more intrinsic characteristics, and have problems for large-scale heterogeneous population. Fuzzy c-means clustering [10], K-means clustering [11], and image histogram are representative algorithms based on raw intensity features, and segmentation results strongly depend on the image quality. However, MRI images suffer from relative intensity scale, inhomogeneous image intensities, and artifacts that degenerate the accuracies of intensity-based algorithms. Shape-based algorithms are sensitive to the shapes of tissues. Due to the high varieties of SAT and VAT among population, the applications of shape-based algorithms may be limited. Graph cut methods [6], [7] have problems of segmenting thin elongated tissues because of the shrinkage bias. Active contour method [8] is prone to be trapped by regions of large gradient magnitude. In short, the above-mentioned algorithms are based on hand-crafted features, which are based on the characteristics of adipose tissues, but sensitive to the noises and hinder their applications to large-scale subject cohorts of different body types. To alleviate this problem, we propose a new deep learning architecture to automatically extract intrinsic features of different tissues. Moreover, considering the spatial distributions of different tissues, we propose new hand-crafted features based on polar coordinates.

In this paper, a two-stage coarse-to-fine algorithm for AAT segmentation is proposed, where the coarse segmentation adopts intrinsic features of different tissues to alleviate effects of various noises, and the fine segmentation considers the spatial consistency for more accurate segmentation. The entire procedure of our framework is shown in Fig. 2. For the first stage, a novel pixel-wise classification algorithm based on discriminative and high-level representations of pixels is presented. First, three types of features, intensity, shape, and contextual features, are included. Shape features are presented by polar coordinates, which are more discriminative than Cartesian coordinates. Contextual features are captured by multi-scale patches centered at each pixel. Second, to explore intrinsic characteristics of tissues, we adopt a novel deep neural network model, called multi-scale deep neural network (MSDNN), to automatically extract the more abstract and high-level representations from these raw features. However, the classification algorithm does not explicitly consider the spatial consistency of tissues, which sometimes misclassifies SAT pixels to VAT pixels, or vice versa. For the second stage, to improve the segmentation accuracy and make the segmentation results more reasonable, we correct the misclassified pixels by deciding the internal boundary of SAT with explicitly spatial consistency constraint. The initialization of internal boundary of SAT is given by a simple binary classifier, and the classification boundaries is determined by the radial coordinates of SAT pixels. Then we prorogate the initializations to other parts by the contagious constraint of boundaries under the Euclidean coordinates, and an iterative algorithm is propose to solve the propagation problem.

The main contributions of this work are as follows:

  • 1.

    We propose a two-stage coarse-to-fine algorithm for AAT segmentation, which simultaneously considers the segmentation accuracy and the spatial consistence. Pixel-wise classification algorithm based on MSDNN is presented for coarse segmentation, and fine segmentation results are obtained by considering the spatial distributions of tissues.

  • 2.

    We propose a novel deep neural network model, MSDNN, which employs multiple parallel networks to extract high-level features from multi-scale inputs. The mini-batch training algorithm allows such a very limited communication between batches, which makes it more appropriate to parallel computing. Therefore, MSDNN can be easily extended to large number of dataset. Besides, a novel location feature is given by considering the special shapes of tissues in our task.

  • 3.

    We perform experiments on a wide range of population with various body types, including normal, overweight, and obese subjects. The experimental results demonstrate the generality of our algorithms. Besides, it takes about 2 s to segment a new 2D abdominal image, which indicates a potential clinical applications.

The rest of this paper is organized as follows. In Section 2 we give a brief review of related work. The proposed algorithm is described in Section 3. Then, we present the data set for evaluation, performance metrics and experiments in Section 4. Finally, discussion and conclusion are in Section 5.

Section snippets

Related work

In this section, we introduce the related work from three aspects: the formation of images used, previous AAT segmentation algorithms, and the recent proposed algorithms applied to medical image processing. Previous automatic algorithms of AAT segmentation are mainly based on computed topography (CT) images [12] and MR images [11], [10], [8], [6], [7]. However, CT exposes the patient to ionizing radiation, limiting its clinical use to diagnosing acute patient illnesses. In our work, we make the

Our approach

In this section, we propose our approach for AAT segmentation, which is formulated as a two-stage coarse-to-fine algorithm. To guarantee the segmentation accuracy on a large-scale population, we not only propose a novel pixel-wise algorithm based on deep neural network for discriminative and intrinsic representations of different tissues, but also take into account the spatial distributions of tissues for fine segmentation. In the first stage, coarse segmentation result is obtained by softmax

Data set

The data for our study is acquired from 150 Chinese volunteers, which includes 60 normal subjects, 60 overweight subjects, and 30 obese subjects. The statistics of the data are presented in Table 1. The abdominal MR images data were obtained from T1 weighted MR scanner (GE Healthcare, Waukesha, WI). 8–10 contiguous axial slices centered at the L4-L5 level of the abdomen were acquired for each volunteer, and the representative one at the umbilicus level was used in our experiments. Each image

Conclusion and future work

In this paper, we aim at segmenting MR images of abdominal adipose tissues to measure the volume of VAT on a large scale subject cohorts. This is achieved by employing MSDNN with multiple hidden layers to extract high-level feature for classification. The input of MSDNN includes intensity features, contextual features, and location and shape features, while the outputs are the tissue label of all pixels. Furthermore, spatial constraint has been imposed on the fine segmentation stage with

Acknowledgement

The authors would like to thank all reviewers for their helpful suggestions and constructive comments. The work is supported by the National Natural Science Foundation of China (Nos. 61202154, 61572316, 61133009), National High-tech R&D Program of China (863 Program) (No. 2015AA015904), the Science and Technology Commission of Shanghai Municipality Program (No. 13511505000), the Interdisciplinary Program of Shanghai Jiao Tong University (No. 14JCY10), the National Natural Science Foundation

Fei Jiang received her B.S. and M.S. degrees from XiDian University in 2008 and 2011. She is now Ph.D candidate at Shanghai Jiao Tong University. Her research interests include deep learning, manifold learning, and medical image analysis.

References (29)

  • S. Horvath, W. Erhart, M. Brosch, O. Ammerpohl, W. von Schönfels, M. Ahrens, N. Heits, J.T. Bell, P.-C. Tsai, T.D....
  • S. Yoshimoto et al.

    Obesity-induced gut microbial metabolite promotes liver cancer through senescence secretome

    Nature

    (2013)
  • Y. Matsushita et al.

    Associations of visceral and subcutaneous fat areas with the prevalence of metabolic risk factor clustering in 6,292 japanese individuals the hitachi health study

    Diabetes Care

    (2010)
  • S.A. Sadananthan et al.

    Automated segmentation of visceral and subcutaneous (deep and superficial) adipose tissues in normal and overweight men

    J. Magn. Reson. Imaging

    (2015)
  • Cited by (19)

    • Multiscale dense convolutional neural network for DSA cerebrovascular segmentation

      2020, Neurocomputing
      Citation Excerpt :

      2D vessel segmentation methods can be further classified as follows: tracking based methods [5,6,7], deformable model based methods [8,9,10] and CNN based methods [11–14]. Because CNN has demonstrated the extraordinary ability of hierarchical learning and real-time prediction compared with non-CNN based methods, a great deal of methods based on CNN have developed rapidly in the related fields of medical image processing [15,16,17], such as CNN-based vessel segmentation [18]. Several studies have already proven that CNN-based methods achieve excellent performance in some segmentation tasks such as retinal vessel segmentation.

    • Systematic Literature Review of Interior Design in Virtual Reality Environment

      2024, Journal of Advanced Research in Applied Sciences and Engineering Technology
    • TBENet:A two-branch boundary enhancement Network for cerebrovascular segmentation

      2023, Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
    View all citing articles on Scopus

    Fei Jiang received her B.S. and M.S. degrees from XiDian University in 2008 and 2011. She is now Ph.D candidate at Shanghai Jiao Tong University. Her research interests include deep learning, manifold learning, and medical image analysis.

    Dr. Huating Li is an Assistant Professor in the School of Medicine at Shanghai Jiao Tong University. She received her PhD in Shanghai Jiao Tong University in 2011 and postdoctoral training in the University of Hong Kong. Dr. Li's research aims to solve questions in the field of clinical experience of endocrinology and metabolism via interdiscipline of medicine and computer science. She has published productively in top-tier journals. Dr. Li is a guest editor of Clinical Science.

    Dr. Xuhong Hou is an associate chief technician of Shanghai Jiao Tong University Affiliated Sixth People's Hospital. She received her PhD in Endocrinology and Metabolism from Shanghai Jiao Tong Unive rsity in 2014, M.P.H. in Epidemiology and Biostatistic from Tianjin Medical University in 1998, a nd M.D. in Public Health from Shanxi Medical University in 1995. Her research interest is mainly in the field of the epidemiology of metabolic diseases. She has published more than 30 papers inte rnationally refereed journals. She is a member of the clinical and preventive group of Chinese Me dical Association and a member of the epidemiology and preventive medicine group of the Chines e diabetes society.

    Bin Sheng received his BA degree in English and BE degree in computer science from Huazhong University of Science and Technology in 2004, and MS degree in software engineering from University of Macau in 2007, and PhD Degree in computer science from The Chinese University of Hong Kong in 2011. He is currently an associate professor in Department of Computer Science and Engineering at Shanghai Jiao Tong University, he also work with Inst. of Software, Chinese Academy of Sciences. His research interests include virtual reality, computer graphics and image based techniques. Dr. Sheng is an Associate Editor for IET Image Processing journal.

    Ruimin Shen received his Bsc and Msc degrees at Dept. of Computer Science from Tsinghua University. He received his PhD degree from Hagen University, Germany. He is currently a professor in Department of Computer Science and Engineering at Shanghai Jiao Tong University. His research interests include virtual reality, computer graphics, E-Learning Technologies; Knowledge Discovery and Data Mining.

    Xiao-Yang Liu received his B.Eng. Degree in computer science from Huazhong University of Science and Technology, China, in 2010. He is currently working toward the PhD degree in the Department of Computer Science and Engineer in Shanghai Jiao Tong University. His research interests include wireless communication, sensor network, and distributed systems.

    Weiping Jia is now Professor of Division of Endocrinology & metabolism at Shanghai Jiao Tong University Affiliated Sixth People's Hospital, Director of Shanghai Clinical Center for Diabetes, Shanghai Key Laboratory for Diabetes and Shanghai Diabetes Institute. Her research interest involved various aspects of diabetes, obesity and metabolic disorders from genetic and molecular biology to disease management, especially the translational research on the disease.

    Ping Li received his Ph.D. from The Chinese University of Hong Kong. He is currently a Lecturer at The Hong Kong Institute of Education. His research interests include image/video stylization, learning analytics, big data visualization, and creative media.

    Dr. Ruogu Fang is an Assistant Professor in the School of Computing and Information Sciences at Florida International University. She received her PhD in Electrical and Computer Engineering from Cornell University in 2014 and B.E in Information Engineering from Zhejiang University in 2009. Dr. Fang's research aims to explore intelligent approaches to bridge the data and medical informatics via machine learning and data mining. She has published productively in top-tier journals and conferences. Dr. Fang is a guest editor of Computerized Medical Imaging and Graphics, organizer of the International Workshop on Sparsity Techniques in Medical Imaging, and a member of the IEEE and ASNR.

    1

    The first three authors contributed equally to this work.

    View full text