Skip to main content
SpringerLink
Log in

The unified extreme learning machines and discriminative random fields for automatic knee cartilage and meniscus segmentation from multi-contrast MR images

  • Special Issue Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Segmenting articular cartilage and meniscus from magnetic resonance (MR) images is an essential task for the assessment of knee pathology. Most of the previous classification-based works for cartilage and meniscus segmentation only rely on independent labellings by a classifier, but do not consider the spatial context interaction. The labels of most image voxels are actually dependent upon their neighbours. In this study, we present an automatic knee segmentation system working on multi-contrast MR images where a novel classification model unifying an extreme learning machine (ELM)-based association potential and a discriminative random field (DRF)-based interaction potential is proposed. The DRF model introduces spatial dependencies between neighbouring voxels to the independent ELM classification. We exploit a rich set of features From multi-contrast MR images to train the proposed classification model and perform the loopy belief propagation for the inference. The proposed model is evaluated on multi-contrast MR datasets acquired from 11 subjects with results outperforming the independent classifiers in terms of segmentation accuracy of both cartilages and menisci.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. Available from: http://www.itksnap.org/pmwiki/pmwiki.php.

  2. Sigmoid function is defined by the formula \(\frac{1}{{1 + \exp ( - t)}}\).

  3. Multi-modal 3D image registration using mutual information maximization: http://www.itk.org/ITK/applications/MutualInfo.html.

  4. Here, the joint model of SVM and DRF, we just use SVM to replace the role of ELM in the proposed unified ELM and DRF model, instead of the original support vector random fields model in [26].

  5. UGM (2011): http://www.di.ens.fr/~mschmidt/Software/UGM.html.

  6. MATLAB codes of ELM algorithm: http://www.ntu.edu.sg/home/egbhuang/ELM_Codes.htm.

References

  1. Bennett, L., Buckland-Wright, J.: Meniscal and articular cartilage changes in knee osteoarthritis: a cross-sectional double-contrast macroradiographic study. Rheumatology 41(8), 917–923 (2002)

    Article  Google Scholar 

  2. Bowers, M., Tung, G., Fleming, B., Crisco, J., Rey, J.: Quantification of meniscal volume by segmentation of 3 T magnetic resonance images. J. Biomech. 40(12), 2811–2815 (2007)

    Article  Google Scholar 

  3. Brau, A., Beatty, P., Skare, S., Bammer, R.: Comparison of reconstruction accuracy and efficiency among autocalibrating data-driven parallel imaging methods. Magn. Reson. Med. 59(2), 382–395 (2008)

    Google Scholar 

  4. Chang, C.C., Lin, C.J.: LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 1–27 (2011) (Software available at http://www.csie.ntu.edu.tw/cjlin/libsvm)

  5. Disler, D., McCauley, T., Kelman, C., Fuchs, M., Ratner, L., Wirth, C., Hospodar, P.: Fat-suppressed three-dimensional spoiled gradient-echo MR imaging of hyaline cartilage defects in the knee: comparison with standard mr imaging and arthroscopy. Am. J. Roentgenol. 167(1), 127–132 (1996)

    Article  Google Scholar 

  6. Dodin, P., Pelletier, J., Martel-Pelletier, J., Abram, F.: Automatic human knee cartilage segmentation from 3-D magnetic resonance images. IEEE Trans. Biomed. Eng. 57(11), 2699–2711 (2010)

    Article  Google Scholar 

  7. Eckstein, F., Cicuttini, F., Raynauld, J., Waterton, J., Peterfy, C.: Magnetic resonance imaging (MRI) of articular cartilage in knee osteoarthritis (OA): morphological assessment. Osteoarthr. Cartil. 14, 46–75 (2006)

    Article  Google Scholar 

  8. Englund, M.: The role of the meniscus in osteoarthritis genesis. Med. Clin. North Am. 93(1), 37–43 (2009)

    Article  Google Scholar 

  9. Folkesson, J., Dam, E., Olsen, O., Pettersen, P., Christiansen, C.: Segmenting articular cartilage automatically using a voxel classification approach. IEEE Trans. Med. Imag. 26(1), 106–115 (2007)

    Google Scholar 

  10. Fripp, J., Bourgeat, P., Engstrom, C., Ourselin, S., Crozier, S., Salvado, O.: Automated segmentation of the menisci from MR images. In: IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 2009. ISBI’09. pp. 510–513. IEEE (2009)

  11. Fripp, J., Crozier, S., Warfield, S., Ourselin, S.: Automatic segmentation and quantitative analysis of the articular cartilages from magnetic resonance images of the knee. IEEE Trans. Med. Imag. 29(1), 55–64 (2010)

    Article  Google Scholar 

  12. Glocker, B., Komodakis, N., Paragios, N., Glaser, C., Tziritas, G., Navab, N.: Primal/dual linear programming and statistical atlases for cartilage segmentation. In: Proc. Int. Conf. Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 536–543 (2007)

  13. Gougoutas, A., Wheaton, A., Borthakur, A., Shapiro, E., Kneeland, J., Udupa, J., Reddy, R.: Cartilage volume quantification via live wire segmentation. Acad. Radiol. 11(12), 1389–1395 (2004)

    Article  Google Scholar 

  14. Grau, V., Mewes, A., Alcaniz, M., Kikinis, R., Warfield, S.: Improved watershed transform for medical image segmentation using prior information. IEEE Trans. Med. Imag. 23(4), 447–458 (2004)

    Article  Google Scholar 

  15. Hata, Y., Kobashi, S., Tokimoto, Y., Ishikawa, M., Ishikawa, H.: Computer aided diagnosis system of meniscal tears with T 1 and T 2 weighted MR images based on fuzzy inference, pp. 55–58. Computational Intelligence, Theory and Applications (2001)

  16. Heimann, T., Morrison, B., Styner, M., Niethammer, M., Warfield, S.: Segmentation of knee images: a grand challenge. In: Proc. Medical Image Analysis for the Clinic: A Grand Challenge, in conjunction with MICCAI 2010, pp. 207–214 (2010)

  17. Huang, G., Wang, D., Lan, Y.: Extreme learning machines: a survey. Int. J. Mach. Learn. Cybern. 2(4), 107–122 (2011)

    Article  Google Scholar 

  18. Huang, G., Zhou, H., Ding, X., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst. Man Cybern. Part B 40(2), 513–529 (2012)

    Google Scholar 

  19. Huang, G., Zhu, Q., Siew, C.: Extreme learning machine: theory and applications. Neurocomputing 70(1–3), 489–501 (2006)

    Article  Google Scholar 

  20. Jaremko, J., Cheng, R., Lambert, R., Habib, A., Ronsky, J.: Reliability of an efficient MRI-based method for estimation of knee cartilage volume using surface registration. Osteoarthr. Cartil. 14(9), 914–922 (2006)

    Article  Google Scholar 

  21. Kapur, T., Beardsley, P., Gibson, S., Grimson, W., Wells, W.: Model-based segmentation of clinical knee MRI. In: Proc. IEEE Intl Workshop on Model-Based 3D Image, Analysis, pp. 97–106 (1998)

  22. König, L., Groher, M., Keil, A., Glaser, C., Reiser, M., Navab, N.: Semi-automatic segmentation of the patellar cartilage in MRI. In: Computer science, Bildverarbeitung für die Medizin (BVM), pp. 404–408 (2007)

  23. Koo, S., Hargreaves, B., Gold, G.: Automatic segmentation of articular cartilage from MRI (2009). U.S. Patent 20,090/306,496

  24. Kumar, S., Hebert, M.: Discriminative random fields. Int. J. Comp. Vision 68(2), 179–201 (2006)

    Article  Google Scholar 

  25. Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proc. Intl. Conf. on Machine Learning (ICML), pp. 282–289 (2001)

  26. Lee, C., Schmidt, M., Murtha, A., Bistritz, A., Sander, J., Greiner, R.: Segmenting brain tumors with conditional random fields and support vector machines. In: Proc. Computer Vision for Biomedical Image Applications, pp. 469–478 (2005)

  27. Lynch J, Zaim S, Zhao J, Stork A, Peterfy C, Genant H (2000) Cartilage segmentation of 3 D MRI scans of the osteoarthritic knee combining user knowledge and active contours. In: Proc. SPIE Med. Imag.: Image Process 3979:925–935

  28. Maurer Jr, C., Qi, R., Raghavan, V.: A linear time algorithm for computing exact euclidean distance transforms of binary images in arbitrary dimensions. IEEE Trans. Pattern Anal. Mach. Intell. 25(2), 265–270 (2003)

    Article  Google Scholar 

  29. Michael, T.: Scientific Computing: an Introductory Survey. McGraw-Hill, New York (2002)

  30. Murphy, K., Weiss, Y., Jordan, M.: Loopy belief propagation for approximate inference: an empirical study. In: Proc. of Uncertainty in AI (UAI), pp. 467–475 (1999)

  31. Pakina, S., Tamez-Pena, J., Tottermanc, S., Parkerd, K.: Segmentation, surface extraction and thickness computation of articular cartilage. Proc. SPIE Med. Imag. Image Process 4684, 155–166 (2002)

  32. Pirnog, C.: Articular cartilage segmentation and tracking in sequential MR images of the knee. Ph.D. thesis, ETH Zurich (2005)

  33. Platt, J., et al.: Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv. Large Margin Classif. 10(3), 61–74 (1999)

    Google Scholar 

  34. Reeder, S., Wen, Z., Yu, H., Pineda, A., Gold, G., Markl, M., Pelc, N.: Multicoil dixon chemical species separation with an iterative least-squares estimation method. Magn. Reson. Med. 51(1), 35–45 (2004)

    Article  Google Scholar 

  35. Sasaki, T., Hata, Y., Ando, Y., Ishikawa, M., Ishikawa, H.: Fuzzy rule-based approach to segment the menisci regions from MR images. In: Proc. SPIE Med. Imag.: Image Process, vol. 3661, pp. 258–265 (1999)

  36. Scheffler, K., Heid, O., Hennig, J.: Magnetization preparation during the steady state: fat-saturated 3 D True FISP. Magn. Reson. Med. 45(6), 1075–1080 (2001)

    Article  Google Scholar 

  37. Seim, H., Kainmueller, D., Lamecker, H., Bindernagel, M., Malinowski, J., Zachow, S.: Model-based auto-segmentation of knee bones and cartilage in MRI data. In: Proc. Medical Image Analysis for the Clinic: A Grand Challenge, in conjunction with MICCAI 2010, pp. 215–223 (2010)

  38. Shapiro, L., Stockman, G.: Computer Vision. Prentice Hall, Englewood Cliffs (2001)

  39. Shim, H., Chang, S., Tao, C., Wang, J., Kwoh, C., Bae, K.: Knee cartilage: efficient and reproducible segmentation on high-spatial-resolution MR images with the semi-automated graph-cut algorithm method. Radiology 251(2), 548–556 (2009)

    Article  Google Scholar 

  40. Solloway, S., Hutchinson, C., Waterton, J., Taylor, C.: The use of active shape models for making thickness measurements of articular cartilage from MR images. Magn. Reson. Med. 37(6), 943–952 (1997)

    Google Scholar 

  41. Stammberger, T., Eckstein, F., Michaelis, M., Englmeier, K., Reiser, M.: Interobserver reproducibility of quantitative cartilage measurements: comparison of B-spline snakes and manual segmentation. Magn. Reson. Imag. 17(7), 1033–1042 (1999)

    Google Scholar 

  42. Swanson, M., Prescott, J., Best, T., Powell, K., Jackson, R., Haq, F., Gurcan, M.: Semi-automated segmentation to assess the lateral meniscus in normal and osteoarthritic knees. Osteoarthr. Cartil. 18(3), 344–353 (2010)

    Article  Google Scholar 

  43. Tamez-Pena, J., Barbu-McInnis, M., Totterman, S.: Knee cartilage extraction and bone-cartilage interface analysis from 3 D MRI data sets. In: Proc. SPIE Med. Imag. Image Process 5370, 1774–1784 (2004)

  44. Tamez-Pena, J., Totterman, S., Parker, K.: Unsupervised statistical segmentation of multispectral volumetric MRI images. Proc. SPIE 3661, 300–311 (1999)

    Article  Google Scholar 

  45. Vapnik, V., Vapnik, V.: Statistical learning theory. Wiley, New York (1998)

    MATH  Google Scholar 

  46. Vincent, G., Wolstenholme, C., Scott, I., Bowes, M.: Fully automatic segmentation of the knee joint using active appearance models. In: Proc. Medical Image Analysis for the Clinic: A Grand Challenge, in conjunction with MICCAI 2010, pp. 224–230 (2010)

  47. Warfield, S., Kaus, M., Jolesz, F., Kikinis, R.: Adaptive, template moderated, spatially varying statistical classification. Med. Image Anal. 4(1), 43–55 (2000)

    Article  Google Scholar 

  48. Yin, Y.: Multi-surface, multi-object optimal image segmentation: Application in three-dimensional knee joint imaged by MR. Ph.D. thesis, The University Of Iowa (2011)

  49. Yin, Y., Zhang, X., Williams, R., Wu, X., Anderson, D., Sonka, M.: LOGISMOS-layered optimal graph image segmentation of multiple objects and surfaces: cartilage segmentation in the knee joint. IEEE Trans. Med. Imag. 29(12), 2023–2037 (2010)

    Article  Google Scholar 

  50. Zhang, K., Lu, W.: Automatic human knee cartilage segmentation from multi-contrast MR images using extreme learning machines and discriminative random fields. In: Proc. Intl. Workshop. on Machine Learning in Medical Imaging, in conjunction with MICCAI, pp. 335–343 (2011)

Download references

Acknowledgments

The authors wish to thanks Seungbum Koo and 11 volunteers from Stanford University for help in acquiring multi-contrast MR scans using a 3T MR system in the Richard M. Lucas Center. The authors are also grateful to Dr. Julio Chacko Kandathil at Tan Tock Seng Hospital, Singapore for manually segmenting the menisci from MR images. K. Zhang also acknowledges the financial support of China Scholarship Council.

Author information

Authors and Affiliations

  1. School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, Singapore

    Kunlei Zhang & Pina Marziliano

  2. Beckman Institute, University of Illinois, Urbana-Champaign, IL, USA

    Wenmiao Lu

Authors
  1. Kunlei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenmiao Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pina Marziliano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kunlei Zhang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhang, K., Lu, W. & Marziliano, P. The unified extreme learning machines and discriminative random fields for automatic knee cartilage and meniscus segmentation from multi-contrast MR images. Machine Vision and Applications 24, 1459–1472 (2013). https://doi.org/10.1007/s00138-012-0466-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-012-0466-9

Keywords

Search

Navigation