Abstract
Computer vision offers many techniques to facilitate the extraction of semantic information from images. If the images include persons, preservation of privacy in computer vision applications is challenging, but undoubtedly desired. A common technique to prevent exposure of identities is to cover peoples’ faces with, for example, a black bar. Although emotions are crucial for reasoning in many applications, facial expressions may be covered, which hinders the recognition of actual emotions. Thus, recorded images containing obfuscated faces may be useless for further analysis and investigation. We introduce an approach that enables automatic detection and obfuscation of faces. To avoid privacy conflicts, we use synthetically generated faces for obfuscation. Furthermore, we reconstruct the facial expressions of the original face, adjust the color of the new face and seamlessly clone it to the original location. To evaluate our approach experimentally, we obfuscate faces from various datasets by applying blurring, pixelation and the proposed technique. To determine the success of obfuscation, we verify whether the original and the resulting face represent the same person using a state-of-the-art matching tool. Our approach successfully obfuscates faces in more than 97% of the cases. This performance is comparable to blurring, which scores around 96%, and even better than pixelation (76%). Moreover, we analyze how effectively emotions can be preserved when obfuscating the faces. For this, we utilize emotion recognizers to recognize the depicted emotions before and after obfuscation. Regardless of the recognizer, our approach preserves emotions more effectively than the other techniques while preserving a convincingly natural appearance.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Agarwal, A., Chattopadhyay, P., Wang, L.: Privacy preservation through facial de-identification with simultaneous emotion preservation. Sig. Image Video Process. 15(5), 951–958 (2020). https://doi.org/10.1007/s11760-020-01819-9
Aggarwal, A., Rathore, R., Chattopadhyay, P., Wang, L.: Epd-net: a gan-based architecture for face de-identification from images. In: 2020 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS) (2020)
Albawi, S., Mohammed, T.A., Al-Zawi, S.: Understanding of a convolutional neural network. In: 2017 International Conference on Engineering and Technology (ICET), pp. 1–6 (2017). https://doi.org/10.1109/ICEngTechnol.2017.8308186
Bergen, N., Labonté, R.: “Everything is perfect, and we have no problems”: detecting and limiting social desirability bias in qualitative research. Qual. Health Res. 30(5), 783–792 (2020)
Bindemann, M., Attard, J., Leach, A., Johnston, R.A.: The effect of image pixelation on unfamiliar-face matching. Appl. Cogn. Psychol. 27(6), 707–717 (2013)
Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: Yolov4: optimal speed and accuracy of object detection (2020)
Boyle, M., Edwards, C., Greenberg, S.: The effects of filtered video on awareness and privacy. In: Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work (2000)
Carrillo, P., Kalva, H., Magliveras, S.: Compression independent reversible encryption for privacy in video surveillance. EURASIP J. Inf. Secur. (2010). https://dl.acm.org/doi/abs/10.5555/2907328.2907469, https://link.springer.com/article/10.1155/2009/429581
Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B., Bharath, A.A.: Generative adversarial networks: an overview. IEEE Sig. Process. Mag. 35(1), 53–65 (2018). https://doi.org/10.1109/MSP.2017.2765202
Deng, L., Sun, W., Xu, D., Ye, Q.: Impact of anonymity on consumers’ online reviews. Psychol. Mark. 38(12), 2259–2270 (2021)
Dufaux, F., Ebrahimi, T.: Scrambling for video surveillance with privacy. In: 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW’06) (2006)
Frome, A., et al.: Large-scale privacy protection in google street view. In: 2009 IEEE 12th International Conference on Computer Vision (2009)
Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 3rd edn. Prentice-Hall Inc, USA (2006)
Gupta, A., Jaiswal, A., Wu, Y., Yadav, V., Natarajan, P.: Adversarial mask generation for preserving visual privacy. In: 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021) (2021)
Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 1510–1519 (2017). https://doi.org/10.1109/ICCV.2017.167
Hukkelås, H., Mester, R., Lindseth, F.: Deepprivacy: a generative adversarial network for face anonymization. In: International Symposium on Visual Computing (2019)
Jain, V., Learned-Miller, E.: Fddb: A benchmark for face detection in unconstrained settings. University of Massachusetts, Amherst, Technical Report (2010)
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of StyleGAN. In: Proceedings CVPR (2020)
Lehrstuhl für Internationales Automobilmanagement, Universität Duisburg-Essen: Taxiladekonzept für Elektrotaxis im öffentlichen Raum. https://talako.uni-due.de/ (2021). Accessed 08 Nov 2021
Li, J., et al.: Identity-preserving face anonymization via adaptively facial attributes obfuscation. In: Proceedings of the 29th ACM International Conference on Multimedia (2021)
Li, T., Choi, M.S.: Deepblur: a simple and effective method for natural image obfuscation. CoRR (2021)
Li, T., Lin, L.: Anonymousnet: natural face de-identification with measurable privacy. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (2019)
Li, Y., Vishwamitra, N., Knijnenburg, B.P., Hu, H., Caine, K.: Blur vs. block: investigating the effectiveness of privacy-enhancing obfuscation for images. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2017)
Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of International Conference on Computer Vision (ICCV) (2015)
Lu, P., Song, B., Xu, L.: Human face recognition based on convolutional neural network and augmented dataset. Syst. Sci. Control Eng. 9(sup2), 29–37 (2021). https://doi.org/10.1080/21642583.2020.1836526
Luan, P., Huynh, V., Tuan Anh, T.: Facial expression recognition using residual masking network. In: IEEE 25th International Conference on Pattern Recognition (2020)
Maximov, M., Elezi, I., Leal-Taixé, L.: Ciagan: conditional identity anonymization generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)
Milder, R., Stam, D., Boons, M.: The effect of anonymity in idea evaluation. Erasmus University, Rotterdam School of Management (2019)
Nagrani, A., Chung, J.S., Zisserman, A.: Voxceleb: a large-scale speaker identification dataset. arXiv preprint arXiv:1706.08612 (2017)
Nguyen, T.A., Coursaris, C.K., Léger, P.M., Sénécal, S., Fredette, M.: Effectiveness of banner ads: An eye tracking and facial expression analysis. In: International Conference on Human-Computer Interaction (2020)
Oswald, D., Sherratt, F., Smith, S.: Handling the hawthorne effect: the challenges surrounding a participant observer. Rev. Soc. Stud. 1(1), 53–73 (2014)
Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. In: ACM SIGGRAPH 2003 Papers (2003)
Saini, M., Atrey, P.K., Mehrotra, S., Emmanuel, S., Kankanhalli, M.: Privacy modeling for video data publication. In: 2010 IEEE International Conference on Multimedia and Expo (2010)
Saini, M., Atrey, P.K., Mehrotra, S., Kankanhalli, M.: Adaptive transformation for robust privacy protection in video surveillance. Adv. MultiMedia 2012 (2012). https://dl.acm.org/doi/abs/10.1155/2012/639649, https://www.hindawi.com/journals/am/2012/639649/
Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015)
Serengil, S.I., Ozpinar, A.: Hyperextended lightface: a facial attribute analysis framework. In: 2021 International Conference on Engineering and Emerging Technologies (ICEET) (2021)
Siarohin, A., Roy, S., Lathuilière, S., Tulyakov, S., Ricci, E., Sebe, N.: Motion-supervised co-part segmentation. In: 2020 25th International Conference on Pattern Recognition (ICPR) (2021)
Stone, A., Potton, A.: Emotional responses to disfigured faces: The influences of perceived anonymity, empathy, and disgust sensitivity. Basic Appl. Soc. Psychol. 36(6), 520–532 (2014). https://doi.org/10.1080/01973533.2014.958491
Sun, Q., Tewari, A., Xu, W., Fritz, M., Theobalt, C., Schiele, B.: A hybrid model for identity obfuscation by face replacement. In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)
Wallbott, H.G.: Bodily expression of emotion. Eur. J. Soc. Psychol. 28, 879–896 (1998)
West, J., Bergstrom, C.: Which face is real? https://www.whichfaceisreal.com/learn.html (2019). Accessed 13 Jul 2022
Wu, Y., Yang, F., Ling, H.: Privacy-protective-gan for face de-identification (2018)
Yang, K., Yau, J., Fei-Fei, L., Deng, J., Russakovsky, O.: A study of face obfuscation in imagenet. CoRR (2021)
Yang, S., Luo, P., Loy, C.C., Tang, X.: Wider face: a face detection benchmark. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Acknowledgment
This research is funded by the Bundesministerium für Wirtschaft und Energie as part of the TALAKO project [20] (grant number 01MZ19002A).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Shahbaz Nejad, B., Roch, P., Handte, M., Marrón, P.J. (2022). Enhancing Privacy in Computer Vision Applications: An Emotion Preserving Approach to Obfuscate Faces. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2022. Lecture Notes in Computer Science, vol 13599. Springer, Cham. https://doi.org/10.1007/978-3-031-20716-7_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-20716-7_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20715-0
Online ISBN: 978-3-031-20716-7
eBook Packages: Computer ScienceComputer Science (R0)