Quantitative and qualitative methods for efficient evaluation of multiple 3D organ segmentations

Volker Dicken; Annika Hänsch; Jan Moltz; Benjamin Haas; Thomas Coradi; Tomasz Morgas; Jan Klein

doi:10.1117/12.2512750

15 March 2019 Quantitative and qualitative methods for efficient evaluation of multiple 3D organ segmentations

Volker Dicken, Annika Hänsch, Jan Moltz, Benjamin Haas, Thomas Coradi, Tomasz Morgas, Jan Klein

Proceedings Volume 10949, Medical Imaging 2019: Image Processing; 1094914 (2019) https://doi.org/10.1117/12.2512750
Event: SPIE Medical Imaging, 2019, San Diego, California, United States

Abstract

Quantitative comparison of automatic results for multi-organ segmentation by means of Dice scores often does not yield satisfactory results. It is especially challenging, when reference contours may be prone to errors. We developed a novel approach that analyzes regions of high mismatch between automatic and reference segmentations. We extract various metrics characterizing these mismatch clusters and compare them to other metrics derived from volume overlap and surface distance histograms by correlating them with qualitative ratings from clinical experts. We show that some novel features based on the mismatch sets or surface distance histograms performed better than the Dice score. We also show how the mismatch clusters can be used to generate visualizations to reduce the workload for visual inspection of segmentation results. The visualizations directly compare reference to automatic result at locations of high mismatch in orthogonal 2D views and 3D scenes zoomed to the appropriate positions. This can make it easier to detect systematic problems of an algorithm or to compare recurrent error patterns for different variants of segmentation algorithms, such as differently parameterized or trained CNN models.

Conference Presentation

Citation Download Citation

Volker Dicken, Annika Hänsch, Jan Moltz, Benjamin Haas, Thomas Coradi, Tomasz Morgas, and Jan Klein "Quantitative and qualitative methods for efficient evaluation of multiple 3D organ segmentations", Proc. SPIE 10949, Medical Imaging 2019: Image Processing, 1094914 (15 March 2019); https://doi.org/10.1117/12.2512750

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available