Skip to main content

Content-Aware Differential Privacy with Conditional Invertible Neural Networks

  • Conference paper
  • First Online:
Distributed, Collaborative, and Federated Learning, and Affordable AI and Healthcare for Resource Diverse Global Health (DeCaF 2022, FAIR 2022)

Abstract

Differential privacy (DP) has arisen as the gold standard in protecting an individual’s privacy in datasets by adding calibrated noise to each data sample. While the application to categorical data is straightforward, its usability in the context of images has been limited. Contrary to categorical data the meaning of an image is inherent in the spatial correlation of neighboring pixels making the simple application of noise infeasible. Invertible Neural Networks (INN) have shown excellent generative performance while still providing the ability to quantify the exact likelihood. Their principle is based on transforming a complicated distribution into a simple one e.g. an image into a spherical Gaussian. We hypothesize that adding noise to the latent space of an INN can enable differentially private image modification. Manipulation of the latent space leads to a modified image while preserving important details. Further, by conditioning the INN on meta-data provided with the dataset we aim at leaving dimensions important for downstream tasks like classification untouched while altering other parts that potentially contain identifying information. We term our method content-aware differential privacy (CADP). We conduct experiments on publicly available benchmarking datasets as well as dedicated medical ones. In addition, we show the generalizability of our method to categorical data. The source code is publicly available at https://github.com/Cardio-AI/CADP.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Abadi, M., et al.: Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (2016). https://doi.org/10.1145/2976749.2978318

  2. Ardizzone, L., Kruse, J., Rother, C., Köthe, U.: Analyzing inverse problems with invertible neural networks. In: International Conference on Learning Representations (2019). https://openreview.net/forum?id=rJed6j0cKX

  3. Ardizzone, L., Lüth, C., Kruse, J., Rother, C., Köthe, U.: Conditional invertible neural networks for guided image generation (2020). https://openreview.net/forum?id=SyxC9TEtPH

  4. Bellovin, S., Dutta, P., Reitlinger, N.: Privacy and synthetic datasets. Stan. Technol. Law Rev. (2018)

    Google Scholar 

  5. Bhadra, S., Kelkar, V.A., Brooks, F.J., Anastasio, M.A.: On hallucinations in tomographic image reconstruction. IEEE Trans. Med. Imaging 40, 3249–3260 (2021)

    Article  Google Scholar 

  6. Bissoto, A., Perez, F., Valle, E., Avila, S.: Skin lesion synthesis with generative adversarial networks. In: OR 2.0 Context-Aware Operating Theaters, Computer Assisted Robotic Endoscopy, Clinical Image-Based Procedures, and Skin Image Analysis, pp. 294–302 (2018)

    Google Scholar 

  7. Dinh, L., Krueger, D., Bengio, Y.: Nice: non-linear independent components estimation. In: International Conference on Learning Representations (2015)

    Google Scholar 

  8. Dinh, L., Sohl-Dickstein, J., Bengio, S.: Density estimation using real NVP. In: International Conference on Learning Representations (2017). https://openreview.net/forum?id=HkpbnH9lx

  9. Dwork, C., Roth, A.: Medical imaging deep learning with differential privacy. Sci. Rep. 11, 1–8 (2021). https://doi.org/10.1038/s41598-021-93030-0

    Article  Google Scholar 

  10. Fan, L.: Image pixelization with differential privacy. In: DBSec (2018)

    Google Scholar 

  11. Frome, A., et al.: Large-scale privacy protection in google street view. In: International Conference on Computer Vision, pp. 2373–2380 (2009). https://doi.org/10.1109/ICCV.2009.5459413

  12. Kermany, D., Zhang, K., Goldbaum, M.: Large dataset of labeled optical coherence tomography (OCT) and chest X-ray images. Cell (2018). https://doi.org/10.17632/rscbjbr9sj.3

  13. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference of Learning Representations (2015)

    Google Scholar 

  14. Kingma, D.P., Dhariwal, P.: Glow: generative flow with invertible \(1\times 1\) convolutions. In: Advances in Neural Information Processing Systems, vol. 31 (2018)

    Google Scholar 

  15. Laves, M.H., Tölle, M., Ortmaier, T.: Uncertainty estimation in medical image denoising with Bayesian deep image prior. In: Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Graphs in Biomedical Image Analysis, pp. 81–96 (2020)

    Google Scholar 

  16. LeCun, Y., Cortes, C., Burges, C.: MNIST handwritten digit database. ATT Labs, vol. 2 (2010). https://yann.lecun.com/exdb/mnist

  17. Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: International Conference on Computer Vision (ICCV), December 2015

    Google Scholar 

  18. McPherson, R., Shokri, R., Shmatikov, V.: Defeating image obfuscation with deep learning (2016)

    Google Scholar 

  19. Oh, S.J., Benenson, R., Fritz, M., Schiele, B.: Faceless person recognition: privacy implications in social media. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 19–35. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_2

    Chapter  Google Scholar 

  20. Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  21. Schütte, A.D., et al.: Overcoming barriers to data sharing with medical image generation: a comprehensive evaluation. NPJ Digit. Med. 4, 1–14 (2021). https://doi.org/10.1038/s41746-021-00507-3

    Article  Google Scholar 

  22. Sorrenson, P., Rother, C., Köthe, U.: Disentanglement by nonlinear ICA with general incompressible-flow networks (GIN). In: International Conference on Learning Representations (2020). https://openreview.net/forum?id=rygeHgSFDH

  23. Usynin, D., et al.: Adversarial interference and its mitigations in privacy-preserving collaborative machine learning. Nat. Mach. Intell. 3(9), 749–758 (2021). https://doi.org/10.1038/s42256-021-00390-3

    Article  Google Scholar 

  24. Waites, C., Cummings, R.: Differentially private normalizing flows for privacy-preserving density estimation. In: AAAI/ACM Conference on AI, Ethics, and Society (2021)

    Google Scholar 

  25. Yoon, J., Jordon, J., van der Schaar, M.: PATE-GAN: generating synthetic data with differential privacy guarantees. In: International Conference on Learning Representations (2019). https://openreview.net/forum?id=S1zk9iRqF7

  26. Ziller, A., Usynin, D., Braren, R., Makowski, M., Rueckert, D., Kaissis, G.: The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci. 9, 211–407 (2014). https://doi.org/10.1561/0400000042

    Article  MathSciNet  Google Scholar 

  27. Ziller, A., Usynin, D., Braren, R., Makowski, M., Rueckert, D., Kaissis, G.: Medical imaging deep learning with differential privacy. Sci. Rep. 11(1), 1–8 (2021). https://doi.org/10.1038/s41598-021-93030-0

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by grants from the Klaus Tschira Foundation within the Informatics for Life framework, by the DZHK (German Centre for Cardiovascular Research), and by the BMBF (German Ministry of Education and Research). The authors gratefully acknowledge the data storage service SDS@hd supported by the Ministry of Science, Research and the Arts Baden-Württemberg (MWK) and the German Research Foundation (DFG) through grant INST 35/1314-1 FUGG and INST 35/1503-1 FUGG.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Malte Tölle .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 692 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tölle, M., Köthe, U., André, F., Meder, B., Engelhardt, S. (2022). Content-Aware Differential Privacy with Conditional Invertible Neural Networks. In: Albarqouni, S., et al. Distributed, Collaborative, and Federated Learning, and Affordable AI and Healthcare for Resource Diverse Global Health. DeCaF FAIR 2022 2022. Lecture Notes in Computer Science, vol 13573. Springer, Cham. https://doi.org/10.1007/978-3-031-18523-6_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-18523-6_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-18522-9

  • Online ISBN: 978-3-031-18523-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics