Skip to main content
Springer Nature Link
Log in

Face Swapping for Solving Collateral Privacy Issues in Multimedia Analytics

  • Conference paper
  • First Online:
MultiMedia Modeling (MMM 2019)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11295))

Included in the following conference series:

  • 2841 Accesses

Abstract

A wide range of components of multimedia analytics systems relies on visual content that is used for supervised (e.g., classification) and unsupervised (e.g., clustering) machine learning methods. This content may contain privacy sensitive information, e.g., show faces of persons. In many cases it is just an inevitable side-effect that persons appear in the content, and the application may not require identification – a situation which we call “collateral privacy issues”. We propose de-identification of faces in images by using a generative adversarial network to generate new face images, and use them to replace faces in the original images. We demonstrate that face swapping does not impact the performance of visual descriptor matching and extraction.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/carpedm20/DCGAN-tensorflow.

  2. 2.

    https://github.com/wuhuikai/FaceSwap.

References

  1. Agarwal, A., Singh, R., Vatsa, M., Noore, A.: Swapped! digital face presentation attack detection via weighted local magnitude pattern. In: 2017 IEEE International Joint Conference on Biometrics (IJCB), pp. 659–665. IEEE (2017)

    Google Scholar 

  2. Ahonen, T., Rahtu, E., Ojansivu, V., Heikkila, J.: Recognition of blurred faces using local phase quantization. In: 19th International Conference on Pattern Recognition, ICPR 2008, pp. 1–4. IEEE (2008)

    Google Scholar 

  3. Badii, A., Einig, M., Piatrik, T., et al.: Overview of the mediaeval 2013 visual privacy task. In: MediaEval (2014)

    Google Scholar 

  4. Bergeron, C., Sidaty, N., Hamidouche, W., Boyadjis, B., Le Feuvre, J., Lim, Y.: Real-time selective encryption solution based on ROI for MPEG-a visual identity management AF. In: 2017 22nd International Conference on Digital Signal Processing (DSP), pp. 1–5, Aug 2017

    Google Scholar 

  5. Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P., Nayar, S.K.: Face swapping: automatically replacing faces in photographs. In: ACM Transactions on Graphics (TOG), vol. 27, p. 39. ACM (2008)

    Article  Google Scholar 

  6. Evaluation framework for compact descriptors for video analysis - search and retrieval - version 2.0. Technical report ISO/IEC JTC1/SC29/WG11/N15729 (2015)

    Google Scholar 

  7. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Official Journal of the European Union, L119:1–88, May 2016

    Google Scholar 

  8. Hadid, A., Nishiyama, M., Sato, Y.: Recognition of blurred faces via facial deblurring combined with blur-tolerant descriptors. In: 2010 20th International Conference on Pattern Recognition (ICPR), pp. 1160–1163. IEEE (2010)

    Google Scholar 

  9. King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)

    Google Scholar 

  10. Lander, K., Bruce, V., Hill, H.: Evaluating the effectiveness of pixelation and blurring on masking the identity of familiar faces. Appl. Cogn. Psychol. 15(1), 101–116 (2001)

    Article  Google Scholar 

  11. Letournel, G., Bugeau, A., Ta, V.T., Domenger, J.P.: Face de-identification with expressions preservation. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 4366–4370. IEEE (2015)

    Google Scholar 

  12. Lin, J., Duan, L.-Y., Huang, Y., Luo, S., Huang, T., Gao, W.: Rate-adaptive compact fisher codes for mobile visual search. IEEE Signal Process. Lett. 21(2), 195–198 (2014)

    Article  Google Scholar 

  13. Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of International Conference on Computer Vision (ICCV) (2015)

    Google Scholar 

  14. Lou, Y., et al.: Compact deep invariant descriptors for video retrieval. In: Data Compression Conference (DCC), pp. 420–429, April 2017

    Google Scholar 

  15. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  MathSciNet  Google Scholar 

  16. Mahajan, S., Chen, L.J., Tsai, T.C.: SwapItUP: a face swap application for privacy protection. In: 2017 IEEE 31st International Conference on Advanced Information Networking and Applications (AINA), pp. 46–50. IEEE (2017)

    Google Scholar 

  17. Meng, L., Sun, Z., Collado, O.T.: Efficient approach to de-identifying faces in videos. IET Sig. Process. 11(9), 1039–1045 (2017)

    Article  Google Scholar 

  18. Nakashima, Y., Babaguchi, N., Fan, J.: Intended human object detection for automatically protecting privacy in mobile video surveillance. Multimed. Syst. 18(2), 157–173 (2012)

    Article  Google Scholar 

  19. Natsume, R., Yatagawa, T., Morishima, S.: RSGAN: face swapping and editing using face and hair representation in latent spaces. arXiv preprint arXiv:1804.03447 (2018)

  20. Newton, E.M., Sweeney, L., Malin, B.: Preserving privacy by de-identifying face images. IEEE Trans. Knowl. Data Eng. 17(2), 232–243 (2005)

    Article  Google Scholar 

  21. Nirkin, Y., Masi, I., Tuan, A.T., Hassner, T., Medioni, G.: On face segmentation, face swapping, and face perception. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition, FG 2018, pp. 98–105. IEEE (2018)

    Google Scholar 

  22. Nishiyama, M., Takeshima, H., Shotton, J., Kozakaya, T., Yamaguchi, O.: Facial deblur inference to improve recognition of blurred faces. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 1115–1122. IEEE (2009)

    Google Scholar 

  23. Padilla-López, J.R., Chaaraoui, A.A., Flórez-Revuelta, F.: Visual privacy protection methods: a survey. Expert. Syst. Appl. 42(9), 4177–4195 (2015)

    Article  Google Scholar 

  24. Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Object retrieval with large vocabularies and fast spatial matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2007)

    Google Scholar 

  25. Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. CoRR, abs/1511.06434 (2015)

    Google Scholar 

  26. Saini, M., Atrey, P.K., Mehrotra, S., Kankanhalli, M.: W3-privacy: understanding what, when, and where inference channels in multi-camera surveillance video. Multimed. Tools Appl. 68(1), 135–158 (2014)

    Article  Google Scholar 

  27. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556 (2014)

    Google Scholar 

  28. Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Sig. Process. Lett. 23(10), 1499–1503 (2016)

    Article  Google Scholar 

  29. Zhang, Y., Zheng, L., Thing, V.L.: Automated face swapping and its detection. In: 2017 IEEE 2nd International Conference on Signal and Image Processing (ICSIP), pp. 15–19. IEEE (2017)

    Google Scholar 

Download references

Acknowledgments

The research leading to these results has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement no 761802, MARCONI (“Multimedia and Augmented Radio Creation: Online, iNteractive, Individual”, https://www.projectmarconi.eu).

Author information

Authors and Affiliations

  1. JOANNEUM RESEARCH Forschungsgesellschaft mbH, DIGITAL – Institute for Information and Communication Technologies, Steyrergasse 17, 8010, Graz, Austria

    Werner Bailer

Authors
  1. Werner Bailer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Werner Bailer .

Editor information

Editors and Affiliations

  1. Information Technologies Institute, Centre for Research and Technology Hellas, Thessaloniki, Greece

    Ioannis Kompatsiaris

  2. EURECOM, Sophia Antipolis, France

    Benoit Huet

  3. Information Technologies Institute, Centre for Research and Technology Hellas, Thessaloniki, Greece

    Vasileios Mezaris

  4. Dublin City University, Dublin, Ireland

    Cathal Gurrin

  5. National Chiao Tung University, Hsinchu, Taiwan

    Wen-Huang Cheng

  6. Information Technologies Institute, Centre for Research and Technology Hellas, Thessaloniki, Greece

    Stefanos Vrochidis

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bailer, W. (2019). Face Swapping for Solving Collateral Privacy Issues in Multimedia Analytics. In: Kompatsiaris, I., Huet, B., Mezaris, V., Gurrin, C., Cheng, WH., Vrochidis, S. (eds) MultiMedia Modeling. MMM 2019. Lecture Notes in Computer Science(), vol 11295. Springer, Cham. https://doi.org/10.1007/978-3-030-05710-7_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-05710-7_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-05709-1

  • Online ISBN: 978-3-030-05710-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation