Abstract
Simulating the disease progression of a suspicious finding in mammography (MG) images could assist in the early detection of breast cancer, where the telltale signs of malignancy are subtle and hard to detect. It could also decrease both unnecessary biopsies and treatments of indolent or low-grade disease that might otherwise remain asymptomatic. We propose a novel approach to simulating disease progression, as a significant step towards those goals. Our architecture uses the powerful Wasserstein GAN in combination with a novel component that simulates the progression of the disease in deep feature space. This allows us to learn from unlabeled longitudinal MG pairs of current and prior studies, stabilize the learning procedure, and overcome misalignment between the MG pairs. Our output image replicates an actual MG image, maintains the prior’s shape and general appearance while also containing a finding with characteristics that resemble the current image’s suspicious finding. We simulate a progression of: (i) a full MG prior image in low-resolution, and (ii) a high-resolution patch in suspicious areas of the prior image. We demonstrate the effectiveness of our pipeline in achieving the above goals using quantitative and qualitative metrics and a reader study. Our results show the high quality of our simulation and the promise it holds for early risk stratification.
M. Raboh, S. Perek—Authors contributed equally.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abdelhafiz, D., Bi, J., Ammar, R., Yang, C., Nabavi, S.: Convolutional neural network for automated mass segmentation in mammography. BMC Bioinform. 21(1), 1–19 (2020)
Arjovsky, M., Bottou, L.: Towards principled methods for training generative adversarial networks. arXiv preprint arXiv:1701.04862 (2017)
Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein gan. arXiv preprint arXiv:1701.07875 (2017)
Borji, A.: Pros and cons of gan evaluation measures. Comput. Vis. Image Underst. 179, 41–65 (2019)
Chen, Y.C., et al.: Facelet-bank for fast portrait manipulation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3541–3549 (2018)
Drozdzal, M., Vorontsov, E., Chartrand, G., Kadoury, S., Pal, C.: The importance of skip connections in biomedical image segmentation. In: Carneiro, G., et al. (eds.) LABELS/DLMIA -2016. LNCS, vol. 10008, pp. 179–187. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46976-8_19
Goodfellow, I., et al.: Generative adversarial nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 2672–2680. Curran Associates, Inc. (2014). https://proceedings.neurips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.: Improved training of wasserstein gans. arXiv preprint arXiv:1704.00028 (2017)
IBM Research, H.: Fusemedml: https://github.com/ibm/fuse-med-ml (2021). https://doi.org/10.5281/ZENODO.5146491, https://zenodo.org/record/5146491
Korkinof, D., Rijken, T., O’Neill, M., Yearsley, J., Harvey, H., Glocker, B.: High-resolution mammogram synthesis using progressive generative adversarial networks. arXiv preprint arXiv:1807.03401 (2018)
Lee, H., Kim, S.T., Lee, J.-H., Ro, Y.M.: Realistic breast mass generation through BIRADS category. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 703–711. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_78
Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)
Monticciolo, D., Helvie, M., Hendrick, R.E.: Current issues in the overdiagnosis and overtreatment of breast cancer. Am. J. Roentgenol. 210, 1–7 (2017). https://doi.org/10.2214/AJR.17.18629
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015)
Ravi, D., Alexander, D.C., Oxtoby, N.P.: Degenerative adversarial NeuroImage nets: generating images that mimic disease progression. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11766, pp. 164–172. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32248-9_19
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Sathyan, A., Martis, D., Cohen, K.: Mass and calcification detection from digital mammograms using unets. In: 2020 7th International Conference on Soft Computing & Machine Intelligence (ISCMI), pp. 229–232. IEEE (2020)
Shen, T., Gou, C., Wang, F.Y., He, Z., Chen, W.: Learning from adversarial medical images for x-ray breast mass segmentation. Comput. Methods Progr. Biomed. 180, 105012 (2019)
Sun, H., et al.: Aunet: attention-guided dense-upsampling networks for breast mass segmentation in whole mammograms. Phys. Med. Biol. 65(5), 055005 (2020)
Upchurch, P., et al.: Deep feature interpolation for image content changes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7064–7073 (2017)
Wegmayr, V., Hörold, M., Buhmann, J.M.: Generative aging of brain MR-images and prediction of Alzheimer progression. In: Fink, G.A., Frintrop, S., Jiang, X. (eds.) DAGM GCPR 2019. LNCS, vol. 11824, pp. 247–260. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33676-9_17
Zhao, Y., Ma, B., Jiang, P., Zeng, D., Wang, X., Li, S.: Prediction of Alzheimer’s disease progression with multi-information generative adversarial network. IEEE J. Biomed. Health Inform. 25(3), 711–719 (2020). https://doi.org/10.1109/JBHI.2020.3006925
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Jubran, I., Raboh, M., Perek, S., Gruen, D., Hexter, E. (2021). A Glimpse into the Future: Disease Progression Simulation for Breast Cancer in Mammograms. In: Svoboda, D., Burgos, N., Wolterink, J.M., Zhao, C. (eds) Simulation and Synthesis in Medical Imaging. SASHIMI 2021. Lecture Notes in Computer Science(), vol 12965. Springer, Cham. https://doi.org/10.1007/978-3-030-87592-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-87592-3_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87591-6
Online ISBN: 978-3-030-87592-3
eBook Packages: Computer ScienceComputer Science (R0)
