HistoContours: a Framework for Visual Annotation of Histopathology Whole Slide Images
View/ Open
Date
2022Author
Al-Thelaya, Khaled
Joad, Faaiz
Gilal, Nauman Ullah
Mifsud, William
Pintore, Giovanni
Agus, Marco
Schneider, Jens
Metadata
Show full item recordAbstract
We present an end-to-end framework for histopathological analysis of whole slide images (WSIs). Our framework uses deep learning-based localization & classification of cell nuclei followed by spatial data aggregation to propagate classes of sparsely distributed nuclei across the entire slide. We use YOLO (''You Only Look Once'') for localization instead of more costly segmentation approaches and show that using HistAuGAN boosts its performance. YOLO finds bounding boxes around nuclei at good accuracy, but the classification accuracy can be improved by other methods. To this end, we extract patches around nuclei from the WSI and consider models from the SqueezeNet, ResNet, and EfficientNet families for classification. Where we do not achieve a clear separation between highest and second-highest softmax activation of the classifier, we use YOLO's output as a secondary vote. The result is a sparse annotation of the WSI, which we turn dense by using kernel density estimation. The result is a full vector of per pixel probabilities for each class of nucleus we consider. This allows us to visualize our results using both color-coding and isocontouring, reducing visual clutter. Our novel nuclei-to-tissue coupling allows histopathologists to work at both the nucleus and the tissue level, a feature appreciated by domain experts in a qualitative user study.
BibTeX
@inproceedings {10.2312:vcbm.20221192,
booktitle = {Eurographics Workshop on Visual Computing for Biology and Medicine},
editor = {Renata G. Raidou and Björn Sommer and Torsten W. Kuhlen and Michael Krone and Thomas Schultz and Hsiang-Yun Wu},
title = {{HistoContours: a Framework for Visual Annotation of Histopathology Whole Slide Images}},
author = {Al-Thelaya, Khaled and Joad, Faaiz and Gilal, Nauman Ullah and Mifsud, William and Pintore, Giovanni and Gobbetti, Enrico and Agus, Marco and Schneider, Jens},
year = {2022},
publisher = {The Eurographics Association},
ISSN = {2070-5786},
ISBN = {978-3-03868-177-9},
DOI = {10.2312/vcbm.20221192}
}
booktitle = {Eurographics Workshop on Visual Computing for Biology and Medicine},
editor = {Renata G. Raidou and Björn Sommer and Torsten W. Kuhlen and Michael Krone and Thomas Schultz and Hsiang-Yun Wu},
title = {{HistoContours: a Framework for Visual Annotation of Histopathology Whole Slide Images}},
author = {Al-Thelaya, Khaled and Joad, Faaiz and Gilal, Nauman Ullah and Mifsud, William and Pintore, Giovanni and Gobbetti, Enrico and Agus, Marco and Schneider, Jens},
year = {2022},
publisher = {The Eurographics Association},
ISSN = {2070-5786},
ISBN = {978-3-03868-177-9},
DOI = {10.2312/vcbm.20221192}
}
Except where otherwise noted, this item's license is described as Attribution 4.0 International License
Related items
Showing items related by title, author, creator and subject.
-
Rational Bézier Guarding
Khanteimouri, Payam; Mandad, Manish; Campen, Marcel (The Eurographics Association and John Wiley & Sons Ltd., 2022)We present a reliable method to generate planar meshes of nonlinear rational triangular elements. The elements are guaranteed to be valid, i.e. defined by injective rational functions. The mesh is guaranteed to conform ... -
VA + Embeddings STAR: A State-of-the-Art Report on the Use of Embeddings in Visual Analytics
Huang, Zeyang; Witschard, Daniel; Kucher, Kostiantyn; Kerren, Andreas (The Eurographics Association and John Wiley & Sons Ltd., 2023)Over the past years, an increasing number of publications in information visualization, especially within the field of visual analytics, have mentioned the term ''embedding'' when describing the computational approach. ... -
Teaching Game Programming in an Upper-level Computing Course Through the Development of a C++ Framework and Middleware
Hooper, Steffan; Wünsche, Burkhard C.; Denny, Paul; Luxton-Reilly, Andrew (The Eurographics Association, 2024)The game development industry has a programming skills shortage, with industry surveys often ranking game programming as the top skill-in-demand across small, mid-sized, and large triple-A (AAA) game studios. C++ programming ...