Skip to main content
SpringerLink
Log in

AFA: adversarial frequency alignment for domain generalized lung nodule detection

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The different reconstruction parameters of CT imaging lead to domain shifts, which limits the generalization of deep learning models and their applications in computer-aided diagnosis systems. In this paper, we investigate the multi-source domain generalization (DG) problem in the context of lung nodule detection from CT images. We first identify the reconstructed convolution kernel as the key parameter leading to domain shifts. Accordingly, we reorganize the public LUNA16 dataset into a domain generalization benchmark, i.e.,, LUNA-DG. Then, we propose a novel DG method by adversarial frequency alignment (AFA). Specifically, we devise an adaptive transition module (ATM) to learn a frequency attention map that can align different domain images in a common frequency domain. For this purpose, a fidelity discriminator and a multi-domain discriminator are used to train the ATM alternately and adversarially. In addition, to mitigate the issue of ineffective gradient back-propagation in naive multi-domain adversarial learning, we propose a novel random domain adversarial learning (RDAL) strategy that can back-propagate effective gradient signals and gradually reduce the gap between multiple domains. The ATM can be combined with nodule detection models through differentiable Fast Fourier Transform (FFT) and inverse FFT, allowing end-to-end training. Experimental results on both LUNA-DG and our in-house datasets validate the superiority of AFA over representative DG methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. LUNA-DG is released on https://github.com/meisun1207/LUNA-DG.

References

  1. Siegel RL, Miller KD, Jemal A (2019) Cancer statistics. CA Cancer J Clin 69(1):7–34

    Article  Google Scholar 

  2. Ding J, Li A, Hu Z, Wang L (2017) Accurate pulmonary nodule detection in computed tomography images using deep convolutional neural networks. Med Image Comput Comput Assist Intervent pp 559–567

  3. Khosravan N, Bagci U (2018) S4nd: single-shot single-scale lung nodule detection. Med Image Comput Comput Assist Intervent pp 794–802

  4. Zhu W, Liu C, Fan W, Xie X (2018) Deeplung: deep 3D dual path nets for automated pulmonary nodule detection and classification. In: IEEE winter conference on applications of computer vision (WACV), pp 673–681

  5. Zhang T, Cheng J, Fu H, Gu Z, Xiao Y, Zhou K et al (2020) Noise adaptation generative adversarial network for medical image analysis. IEEE Trans Med Imag 39(4):1149–1159

    Article  Google Scholar 

  6. Oliveira HN, Ferreira E, Santos JA (2019) Conditional domain adaptation gans for biomedical image segmentation. arXiv preprint arXiv: 1901.05553

  7. Zhang Z, Yang L, Zheng Y (2018) Translating and segmenting multi-modal medical volumes with cycle- and shape-consistency generative adversarial network. IEEE Conf Comput Vis Pattern Recog, pp 9242–9251

  8. Ghifary M, Kleijn WB, Zhang M (2014) Domain adaptive neural networks for object recognition. In: Pacific rim international conference on artificial intelligence, pp 898–904

  9. Long M, Cao Y, Wang J, Jordan MI (2015) Learning transferable features with deep adaptation networks. In: International conference on machine learning (ICML), pp 97–105

  10. Ganin Y, Lempitsky V (2015) Unsupervised domain adaptation by backpropagation. In: International conference on machine learning (ICML), pp 1180–1189

  11. Li H, Pan SJ, Wang S, Kot AC (2018) Domain generalization with adversarial feature learning. In: IEEE conference on computer vision pattern recogintion, pp 5400–5409

  12. Li D, Yang Y, Song YZ, Hospedales T (2018) Learning to generalize: meta-learning for domain generalization. AAAI

  13. Li D, Zhang J, Yang Y, Liu C, Song YZ, Hospedales T (2019) Episodic training for domain generalization. arXiv preprint arXiv: 1902.00113

  14. Nie D, Trullo R, Lian J et al (2018) Medical image synthesis with deep convolutional adversarial networks. IEEE Trans Biomed Eng 65(12):2720–2730

    Article  Google Scholar 

  15. Sun L, Wang J, Huang Y et al (2020) An adversarial learning approach to medical image synthesis for lesion detection. IEEE J Biomed Health Inform 24(8):2303–2314

    Article  Google Scholar 

  16. Yu W, Lei B, Ng MK, et al. (2021) Tensorizing gan with high-order pooling for alzheimer’s disease assessment. IEEE Trans Neural Netw Learn Syst

  17. Wang S, Wang X, Hu Y et al (2020) Diabetic retinopathy diagnosis using multichannel generative adversarial network with semisupervision. IEEE Trans Autom Sci Eng 18(2):574–585

    Article  Google Scholar 

  18. Setio AAA, Traverso A, De Bel T, Berens MS, van den Bogaard C, Cerello P, et al (2017) Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the luna16 challenge. Med Image Anal pp 1–13

  19. Wang J, Lan C, Liu C, Ouyang Y, Qin T (2021) Generalizing to unseen domains: a survey on domain generalization. arXiv preprint arXiv: 2103.03097

  20. Yang Y, Soatto S (2020) Fda: fourier domain adaptation for semantic segmentation. arXiv preprint arXiv: 2004.05498

  21. Dou Q, Castro DC, Kamnitsas K, Glocker B (2019) Domain generalization via model-agnostic learning of semantic features. Adv Neural Inform Process Syst

  22. Zhou K, Yang Y, Qiao Y, Xiang T (2020) Domain adaptive ensemble learning. arXiv preprint arXiv: 2003.07325

  23. Long M, Zhu H, Wang J, Jordan MI (2017) Deep transfer learning with joint adaptation networks. In: International conference on machine learning (ICML), pp 2208–2217

  24. Zhuang F, Cheng X, Luo P, Pan SJ, He Q (2015) Supervised representation learning: transfer learning with deep autoencoders. IJCAI, pp 4119–4125

  25. Zhang Y, Tang H, Jia K, Tan M (2019) Domain-symmetric networks for adversarial domain adaptation. IEEE Conf Comput Vis Pattern Recog, pp 5031–5040

  26. Chen Y, Li W, Sakaridis C, Dai D, Gool LV (2018) Domain adaptive faster r-cnn for object detection in the wild. IEEE Conf Comput Vis Pattern Recog, pp 3339–3348

  27. Zhao H, Zhang S, Wu G, Moura JM, Costeira J P, Jordan GJ (2019) Adversarial multiple source domain adaptation. Adv Neural Inform Process Syst

  28. Liu Y, Wei F, Shao J, Sheng L, Yan J, Wang X (2018) Exploring disentangled feature representation beyond face identification. IEEE Conf Comput Vis Pattern Recog, pp 2080–2089

  29. Li D, Yang Y, Song YZ, Hospedales T (2017) Deeper broader and artier domain generalization. Int Conf Comput Vis

  30. Cordts M, Omran M, Ramos S, Rehfeld T, Enzweiler M, Benenson R, et al (2016) The cityscapes dataset for semantic urban scene understanding. IEEE Conf Comput Vis Pattern Recog, pp 3213–3223

  31. Liu Q, Dou Q, Heng P A (2020) Shape-aware meta-learning for generalizing prostate mri segmentation to unseen domains. Med Image Comput Comput Assist Intervent

  32. Chen C, Bai W, Davies RH, Bhuva AN, Rueckert D (2020) Improving the generalizability of convolutional neural network-based segmentation on cmr images. Front Cardiovasc Med

  33. Gueguen L, Sergeev A, Kadlec B, Liu R, Yosinski J (2018) Faster neural networks straight from jpeg. In: Workshop on neural information processing systems

  34. Ehrlich M, Davis L (2019) Deep residual learning in the jpeg transform domain. Int Conf Comput Vis

  35. Stuchi JA, Boccato L, Attux R (2020) Frequency learning for image classification. arXiv preprint arXiv:2006.15476

  36. Chen Z, Yang H (2020) Manipulated face detector: joint spatial and frequency domain attention network. arXiv preprint arXiv:2005.02958

  37. Tourassi GD, Armato SG, Bergtholdt M, Wiemker R, Klinder T (2016) Pulmonary nodule detection using a cascaded svm classifier. In: SPIE 9785, Medical Imaging 2016: Computer-Aided Diagnosis

  38. Choi WJ, Choi TS (2013) Automated pulmonary nodule detection system in computed tomography images: a hierarchical block classification approach. Entropy pp 507–523

  39. Jacobs C, van Rikxoort EM, Twellmann T, Scholten ET, de Jong PA, Kuhnigk JM et al (2014) Automatic detection of subsolid pulmonary nodules in thoracic computed tomography images. Med Image Anal 18(2):374–384

    Article  Google Scholar 

  40. Hardie RC, Rogers SK, Wilson T, Rogers A (2008) Performance analysis of a new computer aided detection system for identifying lung nodules on chest radiographs. Med Image Anal 12(3):240–258

    Article  Google Scholar 

  41. Ciompi F, de Hoop B, van Riel SJ, Chung K, Scholten ET, Oudkerk M et al (2015) Automatic classification of pulmonary peri-fissural nodules in computed tomography using an ensemble of 2D views and a convolutional neural network out-of-the-box. Med Image Anal 26(1):195–202

    Article  Google Scholar 

  42. Setio AA, Ciompi F, Litjens G, Gerke P, Jacobs C, Van RS et al (2016) Pulmonary nodule detection in ct images: false positive reduction using multi-view convolutional networks. IEEE Trans Med Imag 35(5):1160–1169

    Article  Google Scholar 

  43. Roth HR, Lu L, Liu J, Yao J, Seff A, Cherry K et al (2016) Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans Med Imag 35(5):1170–1181

    Article  Google Scholar 

  44. Ding J, Li A, Hu Z, Wang L (2017) Accurate pulmonary nodule detection in computed tomography images using deep convolutional neural networks. arXiv preprint arXiv:1706.04303

  45. Niemeijer M, Loog M, Abràmoff MD, Viergever MA, Prokop M, van Ginneken B (2011) On combining computer-aided detection systems. IEEE Trans Med Imag 30(2):215–223

  46. Armato SG, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reevesand AP, et al. (2011) The lung image database consortium (lidc) and image database resource initiative (idri): a completed reference database of lung nodules on ct scans. Med Phys pp 915–931

  47. Cormack AM (1963) Representation of a function by its line integrals, with some radiological applications. J Appl Phys 34(9):2722–2727

    Article  Google Scholar 

  48. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. Med Image Comput Comput Assist Intervent pp 234–241

  49. Milletari F, Navab N, Ahmadi S (2016) V-net: fully convolutional neural networks for volumetric medical image segmentation. Int Conf 3D Vis (3DV) pp 565–571

  50. Tzeng E, Hoffman J, Saenko K, Darrell T (2017) Adversarial discriminative domain adaptation. IEEE Conf Comput Vis Pattern Recog

  51. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. IEEE Conf Comput Vis. Pattern Recog, pp 770–778

  52. Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. IEEE Conf Comput Vis Pattern Recog

  53. Li C, Wand M (2016) Precomputed real-time texture synthesis with markovian generative adversarial networks. Eur Conf Comput Vis, pp 702–716

  54. Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. Int Conf Artif Intell Stat, pp 249–256

  55. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. Int Conf Learn Rep

  56. Aberle DR, Adams AM, Berg CD, Black WC, Clapp JD, Fagerstrom RM et al (2011) Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med 365:395–409

    Article  Google Scholar 

  57. Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. Eur Conf Comput Vis, pp 694–711

  58. Howard A, Sandler M, Chen B, Wang W, Chen LC, Tan M, et al (2019) Searching for mobilenetv3. Int Conf Comput Vis

Download references

Acknowledgements

This work was supported by the National Key Research and Development Program of China (2020AAA0107900).

Author information

Authors and Affiliations

  1. University of Science and Technology of China, Hefei, China

    Baocai Yin & Zengfu Wang

  2. iFLYTEK Research, Hefei, China

    Baocai Yin, Mei Sun, Wenchao Liu & Cong Liu

  3. University of Sydney, Darlington, Australia

    Jing Zhang

Authors
  1. Baocai Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mei Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenchao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Cong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zengfu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Baocai Yin.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yin, B., Sun, M., Zhang, J. et al. AFA: adversarial frequency alignment for domain generalized lung nodule detection. Neural Comput & Applic 34, 8039–8050 (2022). https://doi.org/10.1007/s00521-022-06928-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-06928-9

Keywords

Search

Navigation