Abstract
Currently, increasingly large medical imaging data sets become available for research and are analysed by a range of algorithms segmenting anatomical structures automatically and interactively. While they provide segmentations on a much larger scale than possible to achieve with expert annotators, they are typically less accurate than experts. We present and compare approaches to estimate segmentations on large imaging data sets based on a small number of expert annotated examples, and algorithmic segmentations on a much larger data set. Results demonstrate that combining algorithmic segmentations is reliably outperforming the average individual algorithm. Furthermore, injecting organ specific reliability assessments of algorithms based on expert annotations improves accuracy compared to standard label fusion algorithms. The proposed methods are particularly relevant in putting the results of large image analysis algorithm benchmarks to long-term use.
Keywords
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Structures and RadlexIDs: r./l. lungs - RID 1302/1326, liver 58, r./l. kidneys 29662/29663, gallbladder 187, trachea 1247, aorta 480, first lumbar vertebra 29193, r./l. adrenal gland 30324/30325, r./l. psoas major 32248/32249, muscle body of r./l. rectus abdominis 40357/40358, pancreas 170, spleen 86, sternum 2473, urinary bladder 237 and thyroid gland 7578. For Radlex terminology refer to http://www.radlex.org/.
- 2.
Organized by the EU FP7 funded project VISCERAL: http://www.visceral.eu.
- 3.
References
Aljabar, P., Heckemann, R.A., Hammers, A., Hajnal, J.V., Rueckert, D.: Multi-atlas based segmentation of brain images: atlas selection and its effect on accuracy. NeuroImage 46(3), 726–738 (2009)
Artaechevarria, X., Munoz-Barrutia, A., Ortiz-de Solórzano, C.: Combination strategies in multi-atlas image segmentation: application to brain mr data. IEEE Trans. Med. Imaging 28(8), 1266–1277 (2009)
Dice, L.R.: Measures of the amount of ecologic association between species. Ecology 26(3), 297–302 (1945)
Dicente Cid, Y., Depeursinge, A., Jiménez del Toro, O.A., Müller, H.: Efficient and fully automatic segmentation of the lungs in ct volumes. In: Goksel, O., Jiménez-del Toro, O.A., Foncubierta-Rodríguez, A., Müller, H. (eds.) Proceedings of the VISCERAL Challenge at ISBI, vol. 1390, p. 31, April 2015
Gass, T., Szekely, G., Goksel, O.: Multi-atlas segmentation and landmark localization in images with large field of view. In: Menze, B., Langs, G., Montillo, A., Kelm, M., Müller, H., Zhang, S., Cai, W.T., Metaxas, D. (eds.) MCV 2014. LNCS, vol. 8848, pp. 171–180. Springer, Heidelberg (2014)
Göksel, O., Jiménez-del Toro, O.A., Foncubierta-Rodríguez, A., Muller, H.: Overview of the VISCERAL challenge at ISBI 2015. In: Göksel, O., Jiménez-del Toro, O.A., Foncubierta-Rodríguez, A., Müller, H. (eds.) Proceedings of the VISCERAL Challenge at ISBI, New York, NY, May 2015
He, B., Huang, C., Jia, F.: Fully automatic multi-organ segmentation based on multi-boost learning and statistical shape model search. In: Goksel, O., Jiménez-del Toro, O.A., Foncubierta-Rodríguez, A., Müller, H. (eds.) Proceedings of the VISCERAL Challenge at ISBI, vol. 1390, pp. 18–21, April 2015
Heckemann, R.A., Hajnal, J.V., Aljabar, P., Rueckert, D., Hammers, A.: Automatic anatomical brain MRI segmentation combining label propagation and decision fusion. NeuroImage 33(1), 115–126 (2006)
Heinrich, M.P., Maier, O., Handels, H.: Multi-modal multi-atlas segmentation using discrete optimisation and self-similarities. In: Goksel, O., Jiménez-del Toro, O.A., Foncubierta-Rodríguez, A., Müller, H. (eds.) Proceedings of the VISCERAL Challenge at ISBI, vol. 1390, p. 27, April 2015
Jiménez del Toro, O.A., Dicente Cid, Y., Depeursinge, A., Müller, H., Hierarchic anatomical structure segmentation guided by spatial correlations (anatseg-gspac): Visceral anatomy3. In: Goksel, O., Jiménez-del Toro, O.A., Foncubierta-Rodríguez, A., Müller, H. (eds.) Proceedings of the VISCERAL Challenge at ISBI, vol. 1390, pp. 22–66. CEUR-WS, April 2015. http://ceur-ws.org
Jiménez del Toro, O.A., Goksel, O., Menze, B., Müller, H., Langs, G., Weber, M.A., Eggel, I., Gruenberg, K., Holzer, M., Jakab, A., Kotsios-Kontokotsios, G., Krenn, M., Salas Fernandez, T., Schaer, R., Abdel Aziz, T., Winterstein, M., Hanbury, A.: Visceral-visual concept extraction challenge in radiology: ISBI 2014 challenge organization. In: Göksel, O. (ed.) Proceedings of the VISCERAL Challenge at ISBI. CEUR Workshop Proceedings, pp. 6–15 (2014)
Jiménez del Toro, O.A., Müller, H.: Hierarchic Multi–atlas based segmentation for anatomical structures: evaluation in the VISCERAL anatomy benchmarks. In: Menze, B., Langs, G., Montillo, A., Kelm, M., Müller, H., Zhang, S., Cai, W.T., Metaxas, D. (eds.) MCV 2014. LNCS, vol. 8848, pp. 189–200. Springer, Heidelberg (2014)
Kahl, F., Alvén, J., Enqvist, O., Fejne, F., Ulén, J., Fredriksson, J., Landgren, M., Larsson, V.: Good features for reliable registration in multi-atlas segmentation. In: Goksel, O., Jiménez-del Toro, O.A., Foncubierta-Rodríguez, A., Müller, H. (eds.) Proceedings of the VISCERAL Challenge at ISBI, vol. 1390, pp. 12–17, April 2015
Kéchichian, R., Valette, S., Sdika, M., Desvignes, M.: Automatic 3D multiorgan segmentation via clustering and graph cut using spatial relations and hierarchically-registered atlases. In: Menze, B., Langs, G., Montillo, A., Kelm, M., Müller, H., Zhang, S., Cai, W.T., Metaxas, D. (eds.) MCV 2014. LNCS, vol. 8848, pp. 201–209. Springer, Heidelberg (2014)
Kittler, J., Alkoot, F.M.: Sum versus vote fusion in multiple classifier systems. IEEE Trans. Pattern Anal. Mach. Intell. 25(1), 110–115 (2003)
Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20(3), 226–239 (1998)
Klein, S., Staring, M., Murphy, K., Viergever, M., Pluim, J.P.W.: Elastix: a toolbox for intensity-based medical image registration. IEEE Trans. Med. Imaging 29(1), 196–205 (2010)
Langerak, T.R., Van der Heide, U.A., Kotte, A.N.T.J., Viergever, M.A., Van Vulpen, M., Pluim, J.P.W.: Label fusion in atlas-based segmentation using a selective and iterative method for performance level estimation (simple). IEEE Trans. Med. Imaging 29(12), 2000–2008 (2010)
Langs, G., Hanbury, A., Menze, B., Müller, H.: VISCERAL: towards large data in medical imaging — challenges and directions. In: Greenspan, H., Müller, H., Syeda-Mahmood, T. (eds.) MCBR-CDS 2012. LNCS, vol. 7723, pp. 92–98. Springer, Heidelberg (2013)
Li, X., Huang, C., Jia, F., Li, Z., Fang, C., Fan, Y.: Automatic liver segmentation using statistical prior models and free-form deformation. In: Menze, B., Langs, G., Montillo, A., Kelm, M., Müller, H., Zhang, S., Cai, W.T., Metaxas, D. (eds.) MCV 2014. LNCS, vol. 8848, pp. 181–188. Springer, Heidelberg (2014)
Rohlfing, T., Brandt, R., Menzel, R., Maurer, C.R.: Evaluation of atlas selection strategies for atlas-based image segmentation with application to confocal microscopy images of bee brains. NeuroImage 21(4), 1428–1442 (2004)
Roli, F., Kittler, J., Fumera, G., Muntoni, D.: An experimental comparison of classifier fusion rules for multimodal personal identity verification systems. In: Roli, F., Kittler, J. (eds.) MCS 2002. LNCS, vol. 2364, pp. 325–335. Springer, Heidelberg (2002)
Spanier, A.B., Joskowicz, L.: Rule-based ventral cavity multi-organ automatic segmentation in CT scans. In: Menze, B., Langs, G., Montillo, A., Kelm, M., Müller, H., Zhang, S., Cai, W.T., Metaxas, D. (eds.) MCV 2014. LNCS, vol. 8848, pp. 163–170. Springer, Heidelberg (2014)
Studholme, C., Hill, D.L.G., Hawkes, D.J.: An overlap invariant entropy measure of 3D medical image alignment. Pattern recognition 32(1), 71–86 (1999)
Wang, C., Smedby, O.: Automatic multi-organ segmentation using fast model based level set method and hierarchical shape priors. In: Goksel, O., Jiménez-del Toro, O.A., Foncubierta-Rodríguez, A., Müller, H. (eds.) Proceedings of the VISCERAL Challenge at ISBI, vol. 1194, pp. 25–31 (2014)
Warfield, S.K., Zou, K.H., Wells III, W.M.: Validation of image segmentation and expert quality with an expectation-maximization algorithm. In: Dohi, T., Kikinis, R. (eds.) MICCAI 2002, Part I. LNCS, vol. 2488, pp. 298–306. Springer, Heidelberg (2002)
Acknowledgements
The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreements 318068 (VISCERAL) and 257528 (KHRESMOI). We furthermore acknowledge the support of NVIDIA Corporation with the donation of a Tesla K40 GPU used for this work and would like to thank all research groups contributing to this work by participating in the VISCERAL Anatomy 2 & 3 benchmarks [4, 5, 7, 9, 10, 12–14, 20, 23, 25].
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Krenn, M. et al. (2016). Creating a Large-Scale Silver Corpus from Multiple Algorithmic Segmentations. In: Menze, B., et al. Medical Computer Vision: Algorithms for Big Data. MCV 2015. Lecture Notes in Computer Science(), vol 9601. Springer, Cham. https://doi.org/10.1007/978-3-319-42016-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-42016-5_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-42015-8
Online ISBN: 978-3-319-42016-5
eBook Packages: Computer ScienceComputer Science (R0)