Abstract
Face images processed by a biometric system are expected to be used for recognition purposes only. However, recent work presented possibilities for automatically deducing additional information about an individual from their face data. By using soft-biometric estimators, information about gender, age, ethnicity, sexual orientation or the health state of a person can be obtained. This raises a major privacy issue. Previous works presented supervised solutions that require large amount of private data in order to suppress a single attribute. In this work, we propose a privacy-preserving solution that does not require these sensitive information and thus, works in an unsupervised manner. Further, our approach offers privacy protection that is not limited to a single known binary attribute or classifier. We do that by proposing similarity-sensitive noise transformations and investigate their effect and the effect of dimensionality reduction methods on the task of privacy preservation. Experiments are done on a publicly available database and contain analyses of the recognition performance, as well as investigations of the estimation performance of the binary attribute of gender and the continuous attribute of age. We further investigated the estimation performance of these attributes when the prior knowledge about the used privacy mechanism is explicitly utilized. The results show that using this information leads to significantly enhancement of the estimation quality. Finally, we proposed a metric to evaluate the trade-off between the privacy gain and the recognition loss for privacy-preservation techniques. Our experiments showed that the proposed cosine-sensitive noise transformation was successful in reducing the possibility of estimating the soft private information in the data, while having significantly smaller effect on the intended recognition performance.

















Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Bayardo RJ, Agrawal R (2005) Data privacy through optimal k-anonymization. In: 21st International conference on data engineering (ICDE’05), pp 217–228. https://doi.org/10.1109/ICDE.2005.42
Chhabra S, Singh R, Vatsa M, Gupta G (2018) Anonymizing k- facial attributes via adversarial perturbations. CoRR, arXiv:1805.09380
Dantcheva A, Elia P, Ross A (2016) What else does your biometric data reveal? A survey on soft biometrics. IEEE Trans Inf Forens Secur 11(3):441–467. https://doi.org/10.1109/TIFS.2015.2480381. ISSN 1556-6013
Dwork C (2006) Differential privacy. In: Bugliesi M, Preneel B, Sassone V, Wegener I (eds) Automata, languages and programming. Springer, Berlin, pp 1–12
Gross R, Sweeney L, de la Torre F, Baker S (2006) Model-based face de-identification. In: 2006 Conference on computer vision and pattern recognition workshop (CVPRW’06), p 161–161. https://doi.org/10.1109/CVPRW.2006.125
Hyvärinen A, Oja E (2000) Independent component analysis: algorithms and applications. Neural Netw 13(4–5):411–430. https://doi.org/10.1016/S0893-6080(00)00026-5. ISSN 0893-6080
Jourabloo A, Yin X, Liu X (2015) Attribute preserved face de-identification. In: 2015 International conference on biometrics (ICB), pp 278–285. https://doi.org/10.1109/ICB.2015.7139096
King DE (2015) Max-margin object detection. CoRR, arXiv:1502:00046
Li N, Li T, Venkatasubramanian S (2007) t-closeness: privacy beyond k-anonymity and l-diversity. In: 2007 IEEE 23rd international conference on data engineering, pp 106–115. https://doi.org/10.1109/ICDE.2007.367856
Lindell Y, Pinkas B (2000) Privacy preserving data mining. In: Proceedings of the 20th annual international cryptology conference on advances in cryptology, CRYPTO ’00. ISBN 3-540-67907-3. Springer, London, pp 36–54. http://dl.acm.org/citation.cfm?id=646765.704129
Liu W, Wen Y, Yu Z, Li M, Raj B, Le S. (2017) Sphereface: deep hypersphere embedding for face recognition. CoRR, arXiv:1704.08063
Machanavajjhala A, Kifer D, Gehrke J, Venkitasubramaniam M (2007) L-diversity: privacy beyond k-anonymity. ACM Trans Knowl Discov Data 1:1. https://doi.org/10.1145/1217299.1217302. ISSN 1556-4681
Marsaglia G (1972) Choosing a point from the surface of a sphere. Ann Math Statist 43(2):645–646, 04. https://doi.org/10.1214/aoms/1177692644
Mirjalili V, Ross A (2017) Soft biometric privacy: retaining biometric utility of face images while perturbing gender. In: 2017 IEEE International joint conference on biometrics (IJCB), pp 564–573. https://doi.org/10.1109/BTAS.2017.8272743
Mirjalili V, Raschka S, Namboodiri AM, Ross A (2017) Semi-adversarial networks: Convolutional autoencoders for imparting privacy to face images. CoRR, arXiv:1712.00321
Newton EM, Sweeney L, Malin B (2005) Preserving privacy by de-identifying face images. IEEE Trans Knowl Data Eng 17(2):232– 243. https://doi.org/10.1109/TKDE.2005.32. ISSN 1041-4347
Othman A, Ross A (2015) Privacy of facial soft biometrics: suppressing gender but retaining identity. In: Agapito L, Bronstein MM, Rother C (eds) Computer vision - ECCV 2014 workshops. Springer International Publishing, Cham, pp 682–696
Patil H, Kothari A, Bhurchandi K (2016) Expression invariant face recognition using semidecimated dwt, patch-ldsmt, feature and score level fusion. Appl Intell 44(4):913–930. https://doi.org/10.1007/s10489-015-0735-1. ISSN 1573-7497
Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E. (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830
Phillips PJ, Moon H, Rizvi SA, Rauss PJ (2000) The feret evaluation methodology for face-recognition algorithms. IEEE Trans Pattern Anal Mach Intell 22(10):1090–1104. https://doi.org/10.1109/34.879790. ISSN 0162-8828
Rozsa A, Günther M, Rudd EM, Boult TE (2016) Are facial attributes adversarially robust? CoRR, arXiv:1605.05411
Rozsa A, Günther M, Rudd EM, Boult TE (2018) Facial attributes: accuracy and adversarial robustness. CoRR, arXiv:1801.02480
Schölkopf B, Smola AJ, Müller K-R (1999) Advances in kernel methods. Chapter Kernel principal component analysis. MIT Press, Cambridge, pp 327–352. ISBN 0-262-19416-3. http://dl.acm.org/citation.cfm?id=299094.299113
Suo J, Lin L, Shan S, Chen X, Gao W (2011) High-resolution face fusion for gender conversion. IEEE Trans Syst Man Cybern - Part A: Syst Humans 41(2):226–237. https://doi.org/10.1109/TSMCA.2010.2064304. ISSN 1083-4427
Tipping ME, Bishop CM (1999) Probabilistic principal component analysis. J R Statist Soc Series B 61 (3):611–622
Tripathi BK (2017) On the complex domain deep machine learning for face recognition. Appl Intell 47 (2):382–396. https://doi.org/10.1007/s10489-017-0902-7. ISSN 1573-7497
van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9:2579–2605. http://www.jmlr.org/papers/v9/vandermaaten08a.html
Wang C-P, Wei W, Zhang J-S, Song H-B (2018) Robust face recognition via discriminative and common hybrid dictionary learning. Appl Intell 48(1):156–165. https://doi.org/10.1007/s10489-017-0956-6. ISSN 1573-7497
Wang Y, Kosinski M (2018) Deep neural networks are more accurate than humans at detecting sexual orientation from facial images. J Pers Soc Psychol 114(3):246–257
Wei W, Yang X-L, Shen P-Y, Zhou B (2012) Holes detection in anisotropic sensornets: topological methods. Int J Distribd Sensor Netw 8(10):135054. https://doi.org/10.1155/2012/135054
Wei W, Qiang Y, Zhang J (2013) A bijection between lattice-valued filters and lattice-valued congruences in residuated lattices. Mathematical Problems in Engineering
Wei W, Fan X, Song H, Wang H (2017) Video tamper detection based on multi-scale mutual information. Multimed Tools Appl, 1–18
Acknowledgements
This work was supported by the German Federal Ministry of Education and Research (BMBF) as well as by the Hessen State Ministry for Higher Education, Research and the Arts (HMWK) within the Center for Research in Security and Privacy (CRISP). Portions of the research in this paper use the FERET database of facial images collected under the FERET program, sponsored by the DOD Counterdrug Technology Development Program Office.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Terhörst, P., Damer, N., Kirchbuchner, F. et al. Unsupervised privacy-enhancement of face representations using similarity-sensitive noise transformations. Appl Intell 49, 3043–3060 (2019). https://doi.org/10.1007/s10489-019-01432-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-019-01432-5