Localization and Segmentation of 3D Intervertebral Discs from MR Images via a Learning Based Method: A Validation Framework

Abstract

In this paper, we present the results of evaluating our fully automatic intervertebral disc (IVD) localization and segmentation method using the training data and the test data provided by the localization and segmentation challenge organizers of the 3^rd MICCAI Workshop & Challenge on Computational Methods and Clinical Applications for Spine Imaging - MICCAI–CSI2015. We introduce a validation framework consisting of four standard evaluation criteria to evaluate the performance of our method for both localization and segmentation tasks. More specifically, for localization we propose to use the mean localization distance (MLD) with standard deviation (SD) as well as the successful detection rate with three ranges of accuracy. For segmentation, we propose to use the Dice overlap coefficients (DOC) and average absolute distance (AAD) between the automatic segmented disc surfaces and the associated ground truth. Using the proposed metrics, we first validate our previously introduced approach by conducting a comprehensive leave-one-out experiment on the IVD challenge training data which consists of 15 three-dimensional T2-weighted turbo spin echo magnetic resonance (MR) images and the associated ground truth. For localization, we respectively achieved a successful detection rate of 61, 92, and $93\,\%$ when the accuracy range is set to 2.0, 4.0, and 6.0 mm, and a mean localization error of $1.8\,{\pm }\,0.9$ mm. For segmentation, we obtained a mean DOC of $88\,\%$ and a mean AAD of 1.4 mm. We further evaluated the performance of our approach on the test-1 dataset consisting of five MR images released at the pre-test stage and the test-2 dataset consisting of another five MR images released at the on-site competition stage. The results were obtained with a blind test where the performance evaluations were conducted by the challenge organizers. For localization on the test-1 dataset we achieved a successful detection rate of 91.4, 100.0, and $100.0\,\%$ with a MLD$\,{\pm }\,$SD of $1.0\,{\pm }\,0.8$ mm, and for localization on the test-2 dataset we achieved a successful detection rate of 77.1, 100.0, and $100.0\,\%$ with a MLD$\,{\pm }\,$SD of $1.4\,{\pm }\,0.7$ mm, respectively. For segmentation on the test-1 dataset we obtained a mean DOC of $90\,\%$ and a mean AAD of 1.2 mm, and for segmentation on the test-2 dataset we obtained a mean DOC of $92\,\%$ and a mean AAD of 1.3 mm, respectively.