Nicolas Martin
Georges Quénot
ABSTRACT
Segmentation of medical images using learning based systems remains a challenge in medical computer vision: training a segmentation model requires medical images exhaustively annotated by experts that are difficult and expensive to obtain. We propose to explore the usage of partially annotated images, i.e., all images are annotated but not all regions of a given class are annotated. In this paper, we propose several approaches and we experiment them on the segmentation of intra-oral images. First, we propose to modify the loss function to consider only the annotated areas, and second to integrate annotation from non-expert, as well as the combination of these methods. The experiments we conducted showed an improvement up to 33% on the segmentation performance. This approach allows to obtain better quality annotation masks than the initial human annotation using only partially annotated areas or non-expert annotations. In the future, these approaches can be extended by combination with active learning methods.
- John-Melle Bokhorst, Hans Pinckaers, Peter van Zwam, Iris Nagtegaal, Jeroen van der Laak, and Francesco Ciompi. 2019. Learning from sparsely annotated data for semantic segmentation in histopathology images. In Proceedings of The 2nd International Conference on Medical Imaging with Deep Learning. PMLR, 84–91. https://proceedings.mlr.press/v102/bokhorst19a.html ISSN: 2640-3498.Google Scholar
- Yigit B. Can, Krishna Chaitanya, Basil Mustafa, Lisa M. Koch, Ender Konukoglu, and Christian F. Baumgartner. 2018. Learning to Segment Medical Images with Scribble-Supervision Alone. (2018). https://doi.org/10.48550/ARXIV.1807.04668 Publisher: arXiv Version Number: 1.Google Scholar
- Jifeng Dai, Kaiming He, and Jian Sun. 2015. BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation. arXiv:1503.01640 [cs] (May 2015). http://arxiv.org/abs/1503.01640 arXiv:1503.01640.Google Scholar
- Timo Eckhard, E. Valero, and Juan Nieves. 2012. Labial teeth and gingiva color image segmentation for gingival health-state assessment. In Conference on Colour in Graphics.Google Scholar
- Mingchen Gao, Ziyue Xu, Le Lu, Aaron Wu, Isabella Nogues, Ronald M. Summers, and Daniel J. Mollura. 2016. Segmentation label propagation using deep convolutional neural networks and dense conditional random field. In 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI). 1265–1268. https://doi.org/10.1109/ISBI.2016.7493497 ISSN: 1945-8452.Google ScholarCross Ref
- Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2018. Mask R-CNN. arXiv:1703.06870 [cs] (Jan. 2018). http://arxiv.org/abs/1703.06870 arXiv:1703.06870.Google Scholar
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep Residual Learning for Image Recognition. arXiv:1512.03385 [cs] (Dec. 2015). arXiv:1512.03385.Google Scholar
- Mohammad Hesam Hesamian, Wenjing Jia, Xiangjian He, and Paul Kennedy. 2019. Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges. Journal of Digital Imaging 32, 4 (Aug. 2019), 582–596. https://doi.org/10.1007/s10278-019-00227-xGoogle ScholarCross Ref
- Joni Hyttinen, Pauli Fält, Heli Jäsberg, Arja Kullaa, and Markku Hauta-Kasari. 2020. Oral and Dental Spectral Image Database—ODSI-DB. Applied Sciences 10, 20 (Oct. 2020), 7246. https://doi.org/10.3390/app10207246Google ScholarCross Ref
- Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv:1502.03167 [cs] (Feb. 2015). arXiv:1502.03167.Google Scholar
- Shruti Jadon. 2020. A survey of loss functions for semantic segmentation. In 2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB). IEEE, 1–7.Google ScholarDigital Library
- Geert Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindra Adiyoso Setio, Francesco Ciompi, Mohsen Ghafoorian, Jeroen A. W. M. van der Laak, Bram van Ginneken, and Clara I. Sánchez. 2017. A Survey on Deep Learning in Medical Image Analysis. Medical Image Analysis 42 (Dec. 2017), 60–88. https://doi.org/10.1016/j.media.2017.07.005 arXiv:1702.05747.Google Scholar
- Brendon Lutnick, Brandon Ginley, Darshana Govind, Sean D. McGarry, Peter S. LaViolette, Rabi Yacoub, Sanjay Jain, John E. Tomaszewski, Kuang-Yu Jen, and Pinaki Sarder. 2019. An integrated iterative annotation technique for easing neural network training in medical image analysis. Nature Machine Intelligence 1, 2 (Feb. 2019), 112–119. https://doi.org/10.1038/s42256-019-0018-3 Number: 2 Publisher: Nature Publishing Group.Google Scholar
- Shervin Minaee, Yuri Y. Boykov, Fatih Porikli, Antonio J Plaza, Nasser Kehtarnavaz, and Demetri Terzopoulos. 2021. Image Segmentation Using Deep Learning: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021), 1–1. https://doi.org/10.1109/TPAMI.2021.3059968 Conference Name: IEEE Transactions on Pattern Analysis and Machine Intelligence.Google ScholarCross Ref
- Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2016. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv:1506.01497 [cs] (Jan. 2016). http://arxiv.org/abs/1506.01497 arXiv:1506.01497.Google Scholar
- Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv:1505.04597 [cs] (May 2015). http://arxiv.org/abs/1505.04597 arXiv:1505.04597.Google Scholar
- Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. 2015. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV) 115, 3 (2015), 211–252. https://doi.org/10.1007/s11263-015-0816-yGoogle ScholarDigital Library
- Nima Tajbakhsh, Laura Jeyaseelan, Qian Li, Jeffrey N. Chiang, Zhihao Wu, and Xiaowei Ding. 2020. Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation. Medical Image Analysis 63 (July 2020), 101693. https://doi.org/10.1016/j.media.2020.101693Google Scholar
- Maxim Tkachenko, Mikhail Malyuk, Nikita Shevchenko, Andrey Holmanyuk, and Nikolai Liubimov. 2020-2021. Label Studio: Data labeling software. Open source software available from https://github.com/heartexlabs/label-studio.Google Scholar
- Özgün Çiçek, Ahmed Abdulkadir, Soeren S. Lienkamp, Thomas Brox, and Olaf Ronneberger. 2016. 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. arXiv:1606.06650 [cs] (June 2016). http://arxiv.org/abs/1606.06650 arXiv:1606.06650.Google Scholar
Index Terms
- Segmenting partially annotated medical images
Recommendations
Mining partially annotated images
KDD '11: Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data miningIn this paper, we study the problem of mining partially annotated images. We first define what the problem of mining partially annotated images is, and argue that in many real-world applications annotated images are typically partially annotated and ...
Segmentation of CT Brain Images Using Unsupervised Clusterings
In this paper, we present non-identical unsupervised clustering techniques for the segmentation of CT brain images. Prior to segmentation, we enhance the visualization of the original image. Generally, for the presence of abnormal regions in the brain ...
A multi-objective segmentation method for chest X-rays based on collaborative learning from multiple partially annotated datasets
AbstractAccurate segmentation of multiple targets, such as ribs, clavicles, heart, and lung fields, from chest X-ray images is crucial for diagnosing various lung diseases. Currently, mainstream deep learning methods heavily rely on fully annotated large-...
Highlights- This is the multi-organ segmentation framework for chest X-rays.
- It stands out by collaborative learning on multiple partially annotated datasets.
- We construct a shared encoder using densely connected blocks.
- It incorporates a ...
Comments