Skip to main content
SpringerLink
Log in

Rethinking Pulmonary Nodule Detection in Multi-view 3D CT Point Cloud Representation

  • Conference paper
  • First Online:
Machine Learning in Medical Imaging (MLMI 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12966))

Included in the following conference series:

  • 3989 Accesses

Abstract

3D CT point clouds reconstructed from the original CT images are naturally represented in real-world coordinates. Compared with CT images, 3D CT point clouds contain invariant geometric features with irregular spatial distributions from multiple viewpoints. This paper rethinks pulmonary nodule detection in CT point cloud representations. We first extract the multi-view features from a sparse convolutional (SparseConv) encoder by rotating the point clouds with different angles in the world coordinate. Then, to simultaneously learn the discriminative and robust spatial features from various viewpoints, a nodule proposal optimization schema is proposed to obtain coarse nodule regions by aggregating consistent nodule proposals prediction from multi-view features. Last, the multi-level features and semantic segmentation features extracted from a SparseConv decoder are concatenated with multi-view features for final nodule region regression. Experiments on the benchmark dataset (LUNA16) demonstrate the feasibility of applying CT point clouds in lung nodule detection task. Furthermore, we observe that by combining multi-view predictions, the performance of the proposed framework is greatly improved compared to single-view, while the interior texture features of nodules from images are more suitable for detecting nodules in small sizes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aaa, S., et al.: Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge. Med. Image Anal. 42, 1–13 (2017). [dataset]

    Google Scholar 

  2. Ahmed, S.M., Liang, P., Chew, C.M.: EPN: edge-aware PointNet for object recognition from multi-view 2.5 D point clouds. In: IROS, pp. 3445–3450 (2019)

    Google Scholar 

  3. Bandos, A.I., Rockette, H.E., Song, T., Gur, D.: Area under the free-response ROC curve (FROC) and a related summary index. Biometrics 65(1), 247–256 (2009)

    Article  MathSciNet  Google Scholar 

  4. Chen, R., Han, S., Xu, J., Su, H.: Point-based multi-view stereo network. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1538–1547 (2019)

    Google Scholar 

  5. Ding, J., Li, A., Hu, Z., Wang, L.: Accurate pulmonary nodule detection in computed tomography images using deep convolutional neural networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 559–567. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66179-7_64

    Chapter  Google Scholar 

  6. Dou, Q., Chen, H., Jin, Y., Lin, H., Qin, J., Heng, P.A.: Automated pulmonary nodule detection via 3D convnets with online sample filtering and hybrid-loss residual learning. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 630–638 (2017)

    Google Scholar 

  7. Dou, Q., Chen, H., Yu, L., Qin, J., Heng, P.A.: Multilevel contextual 3-D CNNs for false positive reduction in pulmonary nodule detection. IEEE Trans. Biomed. Eng. 64(7), 1558–1567 (2017)

    Article  Google Scholar 

  8. Drokin, I., Ericheva, E.: Deep learning on point clouds for false positive reduction at nodule detection in chest CT scans. arXiv preprint arXiv:2005.03654 (2020)

  9. El-Regaily, S.A., Salem, M.A.M., Aziz, M.H.A., Roushdy, M.I.: Multi-view convolutional neural network for lung nodule false positive reduction. Expert Syst. Appl. 162, 113017 (2020)

    Google Scholar 

  10. Gong, Z., Li, D., Lin, J., Zhang, Y., Lam, K.M.: Towards accurate pulmonary nodule detection by representing nodules as points with high-resolution network. IEEE Access 8, 157391–157402 (2020)

    Article  Google Scholar 

  11. Gupta, A., Saar, T., Martens, O., Moullec, Y.L.: Automatic detection of multisize pulmonary nodules in CT images: large-scale validation of the false-positive reduction step. Med. Phys. 45(3), 1135–1149 (2018)

    Article  Google Scholar 

  12. Han, Z., Wang, X., Liu, Y.S., Zwicker, M.: Multi-angle point cloud-VAE: unsupervised feature learning for 3D point clouds from multiple angles by joint self-reconstruction and half-to-half prediction. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10441–10450. IEEE (2019)

    Google Scholar 

  13. Khosravan, N., Bagci, U.: S4ND: single-shot single-scale lung nodule detection. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 794–802 (2018)

    Google Scholar 

  14. Kim, B.C., Yoon, J.S., Choi, J.S., Suk, H.I.: Multi-scale gradual integration CNN for false positive reduction in pulmonary nodule detection. Neural Netw. (2019)

    Google Scholar 

  15. Li, Y., Fan, Y.: DeepSEED: 3D squeeze-and-excitation encoder-decoder convolutional neural networks for pulmonary nodule detection. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 1866–1869. IEEE (2020)

    Google Scholar 

  16. Li, Z., Zhang, S., Zhang, J., Huang, K., Wang, Y., Yu, Y.: MVP-Net: multi-view FPN with position-aware attention for deep universal lesion detection. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 13–21. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_2

    Chapter  Google Scholar 

  17. Liu, J., Cao, L., Akin, O., Tian, Y.: 3DFPN-HS\(^2\): 3D feature pyramid network based high sensitivity and specificity pulmonary nodule detection. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 513–521. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_57

    Chapter  Google Scholar 

  18. Liu, J., Cao, L., Akin, O., Tian, Y.: Accurate and robust pulmonary nodule detection by 3D feature pyramid network with self-supervised feature learning. arXiv preprint arXiv:1907.11704 (2019)

  19. Phan, A.V., Le Nguyen, M., Nguyen, Y.L.H., Bui, L.T.: DGCNN: a convolutional neural network over large-scale labeled graphs. Neural Netw. 108, 533–543 (2018)

    Article  Google Scholar 

  20. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652–660 (2017)

    Google Scholar 

  21. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems, pp. 5099–5108 (2017)

    Google Scholar 

  22. Setio, A.A.A., et al.: Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans. Med. Imaging 35(5), 1160–1169 (2016). https://doi.org/10.1109/TMI.2016.2536809

    Article  Google Scholar 

  23. Setio, A.A.A., et al.: Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans. Med. Imaging 35(5), 1160–1169 (2016)

    Article  Google Scholar 

  24. Shi, S., et al.: PV-RCNN: point-voxel feature set abstraction for 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10529–10538 (2020)

    Google Scholar 

  25. Shi, S., Wang, Z., Shi, J., Wang, X., Li, H.: From points to parts: 3D object detection from point cloud with part-aware and part-aggregation network. IEEE Trans. Pattern Anal. Mach. Intell. (2020)

    Google Scholar 

  26. Siegel, R.L., Miller, K.D., Jemal, A.: Cancer statistics, 2019. CA Cancer J. Clin. 69(1), 7–34 (2019)

    Google Scholar 

  27. Team, O.D.: OpenPCDet: an open-source toolbox for 3D object detection from point clouds (2020). https://github.com/open-mmlab/OpenPCDet

  28. Usman, M., Lee, B.D., Byon, S.S., Kim, S.H., Lee, B.i., Shin, Y.G.: Volumetric lung nodule segmentation using adaptive ROI with multi-view residual learning. Sci. Rep. 10(1), 1–15 (2020)

    Google Scholar 

  29. Wang, B., Qi, G., Tang, S., Zhang, L., Deng, L., Zhang, Y.: Automated pulmonary nodule detection: high sensitivity with few candidates. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 759–767. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_84

    Chapter  Google Scholar 

  30. Zanjani, F.G., Moin, D.A., Verheij, B., Claessen, F., Cherici, T., Tan, T., et al.: Deep learning approach to semantic segmentation in 3D point cloud intra-oral scans of teeth. In: International Conference on Medical Imaging with Deep Learning, pp. 557–571 (2019)

    Google Scholar 

  31. Zhu, W., Liu, C., Fan, W., Xie, X.: DeepLung: deep 3D dual path nets for automated pulmonary nodule detection and classification. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 673–681 (2018)

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Science Foundation under award number IIS-2041307 and Memorial Sloan Kettering Cancer Center Support Grant/Core Grant P30 CA008748.

Author information

Authors and Affiliations

  1. The City College of New York, New York, NY, 10031, USA

    Jingya Liu & Yingli Tian

  2. Memorial Sloan Kettering Cancer Center, New York, NY, 10065, USA

    Oguz Akin

Authors
  1. Jingya Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Oguz Akin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yingli Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yingli Tian .

Editor information

Editors and Affiliations

  1. Xi'an Jiaotong University, Xi'an, China

    Chunfeng Lian

  2. United Imaging Intelligence, Shanghai, China

    Xiaohuan Cao

  3. Istanbul Technical University, Istanbul, Turkey

    Islem Rekik

  4. Rensselaer Polytechnic Institute, Troy, NY, USA

    Xuanang Xu

  5. Rensselaer Polytechnic Institute, Troy, NY, USA

    Pingkun Yan

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liu, J., Akin, O., Tian, Y. (2021). Rethinking Pulmonary Nodule Detection in Multi-view 3D CT Point Cloud Representation. In: Lian, C., Cao, X., Rekik, I., Xu, X., Yan, P. (eds) Machine Learning in Medical Imaging. MLMI 2021. Lecture Notes in Computer Science(), vol 12966. Springer, Cham. https://doi.org/10.1007/978-3-030-87589-3_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-87589-3_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-87588-6

  • Online ISBN: 978-3-030-87589-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation