Abstract
Pulmonary nodule detection from lung computed tomography (CT) scans has been an active clinical research direction, benefiting the early diagnosis of lung cancer related disease. However, state-of-the-art deep learning models require instance-level annotation for the training data (i.e., a bounding box for each nodule), which require expensive costs and might not always be applicable. On the other hand, during clinical diagnosis of lung nodule detection, radiologists provide electronic medical records (EMR), which contain information such as the malignancy, number, texture of the detected nodules, and slice indices at which the nodules are located. Thus, the goal of this work is to utilize EMR information for learning pulmonary nodule detection models, without observing any nodule annotation during the training stage. To realize the above weakly supervised learning strategy, we extend multiple instance learning (MIL) and specifically take the presence and number of nodules in each CT scan, as well as the associated slice information, in our proposed deep learning framework. In our experiments, we present proper evaluation metrics for assessing and comparing the effectiveness of state-of-the-art models on multiple datasets, which verify the practicality of our proposed model.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
References
Bilen, H., Vedaldi, A.: Weakly supervised deep detection networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Cao, H., et al.: A two-stage convolutional neural networks for lung nodule detection. IEEE Journal of Biomedical and Health Informatics (2020)
Chikontwe, P., Kim, M., Nam, S.J., Go, H., Park, S.H.: Multiple instance learning with center embeddings for histopathology classification. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (2020)
Ding, J., Li, A., Hu, Z., Wang, L.: Accurate pulmonary nodule detection in computed tomography images using deep convolutional neural networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) (2017)
Dou, Q., Chen, H., Jin, Y., Lin, H., Qin, J., Heng, P.: Automated pulmonary nodule detection via 3d convnets with online sample filtering and hybrid-loss residual learning. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) (2017)
Dou, Q., Chen, H., Yu, L., Qin, J., Heng, P.: Multilevel contextual 3-d cnns for false positive reduction in pulmonary nodule detection. IEEE Trans. Biomed. Eng. 64(7), 1558–1567 (2017)
Gao, M., Li, A., Yu, R., Morariu, V.I., Davis, L.S.: C-wsl: count-guided weakly supervised localization. In: European Conference on Computer Vision (ECCV) (2018)
Girshick, R.B.: Fast R-CNN. In: IEEE International Conference on Computer Vision (ICCV) (2015)
Hamidian, S., Sahiner, B., Petrick, N., Pezeshk, A.: 3D convolutional neural network for automatic detection of lung nodules in chest CT. In: Medical Imaging 2017: Computer-Aided Diagnosis (2017)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
Ilse, M., Tomczak, J., Welling, M.: Attention-based deep multiple instance learning. In: International Conference on Machine Learning (ICML) (2018)
Khosravan, N., Bagci, U.: S4ND: single-shot single-scale lung nodule detection. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) (2018)
Liu, J., Cao, L., Akin, O., Tian, Y.: Accurate and robust pulmonary nodule detection by 3D feature pyramid network with self-supervised feature learning. arXiv preprint arXiv:1907.11704 (2019)
Ren, S., He, K., Girshick, R.B., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems (NeurIPS) (2015)
Ren, Z., et al.: Instance-aware, context-focused, and memory-efficient weakly supervised object detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Sadafi, A., et al.: Attention based multiple instance learning for classification of blood cell disorders. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) (2020)
Setio, A.A.A., et al.: Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the luna16 challenge. Med. Image Anal. 42, 1–13 (2017)
Tang, P., Wang, X., Bai, X., Liu, W.: Multiple instance detection network with online instance classifier refinement. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
Wan, F., Liu, C., Ke, W., Ji, X., Jiao, J., Ye, Q.: C-MIL: continuation multiple instance learning for weakly supervised object detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Wang, B., Qi, G., Tang, S., Zhang, L., Deng, L., Zhang, Y.: Automated pulmonary nodule detection: high sensitivity with few candidates. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) (2018)
Wang, D., Zhang, Y., Zhang, K., Wang, L.: Focalmix: semi-supervised learning for 3D medical image detection. In: CVPR (2020)
Wang, Q., Shen, F., Shen, L., Huang, J., Sheng, W.: Lung nodule detection in CT images using a raw patch-based convolutional neural network (2019)
Xu, G., et al.: Camel: a weakly supervised learning framework for histopathology image segmentation. In: IEEE International Conference on Computer Vision (ICCV) (2019)
Yan, G., et al.: C-MIDN: coupled multiple instance detection network with segmentation guidance for weakly supervised object detection. In: IEEE International Conference on Computer Vision (ICCV) (2019)
Yang, C.H., Qi, J., Chen, P.Y., Ma, X., Lee, C.H.: Characterizing speech adversarial examples using self-attention u-net enhancement. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2020)
Yang, C.H.H., Siniscalchi, S.M., Lee, C.H.: Pate-aae: Incorporating adversarial autoencoder into private aggregation of teacher ensembles for spoken command classification. arXiv preprint arXiv:2104.01271 (2021)
Yang, H.H., Huang, K.C., Chen, W.T.: LAFFNet: a lightweight adaptive feature fusion network for underwater image enhancement. In: IEEE International Conference on Robotics and Automation (ICRA) (2021)
Yang, H.H., Yang, C.H.H., Wang, Y.C.F.: Wavelet channel attention module with a fusion network for single image deraining. In: IEEE International Conference on Image Processing (ICIP) (2020)
Zeng, Z., Liu, B., Fu, J., Chao, H., Zhang, L.: WSOD2: learning bottom-up and top-down objectness distillation for weakly-supervised object detection. In: IEEE International Conference on Computer Vision (ICCV) (2019)
Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: Beyond empirical risk minimization. International Conference on Learning Representations (ICLR) (2018)
Zheng, S., Guo, J., Cui, X., Veldhuis, R.N.J., Oudkerk, M., van Ooijen, P.M.A.: Automatic pulmonary nodule detection in CT scans using convolutional neural networks based on maximum intensity projection. IEEE Transactions on Medical Imaging (2020)
Zhu, W., Liu, C., Fan, W., Xie, X.: Deeplung: deep 3D dual path nets for automated pulmonary nodule detection and classification. In: IEEE Winter Conference on Applications of Computer Vision (2018)
Zhu, W., Vang, Y.S., Huang, Y., Xie, X.: Deepem: deep 3D convnets with em for weakly supervised pulmonary nodule detection. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) (2018)
Acknowledgements
This project is partly funded by Ministry of Science and Technology of Taiwan (MOST 110-2634-F-002-036).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, HH. et al. (2021). Leveraging Auxiliary Information from EMR for Weakly Supervised Pulmonary Nodule Detection. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12907. Springer, Cham. https://doi.org/10.1007/978-3-030-87234-2_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-87234-2_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87233-5
Online ISBN: 978-3-030-87234-2
eBook Packages: Computer ScienceComputer Science (R0)
