Skip to main content
SpringerLink
Log in

Privacy-Aware Face Recognition with Lensless Multi-pinhole Camera

  • Conference paper
  • First Online:
Computer Vision – ECCV 2020 Workshops (ECCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12539))

Included in the following conference series:

Abstract

Face recognition and privacy protection are closely related. A high-quality facial image is required to achieve a high accuracy in face recognition; however, this undermines the privacy of the person being photographed. From the perspective of confidentiality, storing facial images as raw data is a problem. If a low-quality facial image is used, to protect user privacy, the accuracy of recognition decreases. In this paper, we propose a method for face recognition that solves these problems. We train a neural network with an unblurred image at first, and then train the neural network with a blurred image, using the features of the neural network trained with the unblurred image, as an initial value. This makes it possible to train features that are similar to the features trained with the neural network using a high-quality image. This enables us to perform face recognition without compromising user privacy. Our method consists of a neural network for face feature extraction, which extracts suitable features for face recognition from a blurred facial image, and a face recognition neural network. After pretraining both networks, we fine-tune them in an end-to-end manner. In experiments, the proposed method achieved accuracy comparable to that of conventional face recognition methods, which take as input unblurred face images from simulations and from images captured by our camera system.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Akiba, T., Sano, S., Yanase, T., Ohta, T., Koyama, M.: Optuna: A next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2623–2631 (2019)

    Google Scholar 

  2. Asif, M.S., Ayremlou, A., Sankaranarayanan, A., Veeraraghavan, A., Baraniuk, R.G.: Flatcam: thin, lensless cameras using coded aperture and computation. IEEE Trans. Comput. Imag. 3(3), 384–397 (2016)

    Article  MathSciNet  Google Scholar 

  3. Best-Rowden, L., Bisht, S., Klontz, J.C., Jain, A.K.: Unconstrained face recognition: Establishing baseline human performance via crowdsourcing. In: IEEE International Joint Conference on Biometrics, pp. 1–8. IEEE (2014)

    Google Scholar 

  4. Browarek, S.: High resolution, Low cost, Privacy preserving Human motion tracking System via passive thermal sensing. Ph.D. thesis, Massachusetts Institute of Technology (2010)

    Google Scholar 

  5. Canh, T.N., Nagahara, H.: Deep compressive sensing for visual privacy protection in flatcam imaging. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 3978–3986. IEEE (2019)

    Google Scholar 

  6. Cannon, T., Fenimore, E.: Tomographical imaging using uniformly redundant arrays. Appl. Opt. 18(7), 1052–1057 (1979)

    Article  Google Scholar 

  7. Chen, B.C., Chen, C.S., Hsu, W.H.: Face recognition and retrieval using cross-age reference coding with cross-age celebrity dataset. IEEE Trans. Multimedia 17(6), 804–815 (2015)

    Article  Google Scholar 

  8. Chen, R., Mihaylova, L., Zhu, H., Bouaynaya, N.C.: A deep learning framework for joint image restoration and recognition. In: Circuits, Systems, and Signal Processing, pp. 1–20 (2019)

    Google Scholar 

  9. Chrysos, G.G., Zafeiriou, S.: Deep face deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 69–78 (2017)

    Google Scholar 

  10. Cossalter, M., Tagliasacchi, M., Valenzise, G.: Privacy-enabled object tracking in video sequences using compressive sensing. In: 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance, pp. 436–441. IEEE (2009)

    Google Scholar 

  11. Dai, J., Wu, J., Saghafi, B., Konrad, J., Ishwar, P.: Towards privacy-preserving activity recognition using extremely low temporal and spatial resolution cameras. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 68–76 (2015)

    Google Scholar 

  12. Deng, J., Guo, J., Xue, N., Zafeiriou, S.: Arcface: additive angular margin loss for deep face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4690–4699 (2019)

    Google Scholar 

  13. Fernandes, F.E., Yang, G., Do, H.M., Sheng, W.: Detection of privacy-sensitive situations for social robots in smart homes. In: 2016 IEEE International Conference on Automation Science and Engineering (CASE), pp. 727–732. IEEE (2016)

    Google Scholar 

  14. Gade, R., Moeslund, T.B.: Thermal cameras and applications: a survey. Mach. Vis. Appl. 25(1), 245–262 (2014)

    Article  Google Scholar 

  15. Gallego, G., et al.: Event-based vision: A survey. arXiv preprint arXiv:1904.08405 (2019)

  16. Gupta, K., Bhowmick, B., Majumdar, A.: Motion blur removal via coupled autoencoder. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 480–484. IEEE (2017)

    Google Scholar 

  17. Hiura, S., Matsuyama, T.: Depth measurement by the multi-focus camera. In: Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No. 98CB36231), pp. 953–959. IEEE (1998)

    Google Scholar 

  18. Huang, G.B., Mattar, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: A database forstudying face recognition in unconstrained environments (2008)

    Google Scholar 

  19. Inagaki, Y., Kobayashi, Y., Takahashi, K., Fujii, T., Nagahara, H.: Learning to capture light fields through a coded aperture camera. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 418–434 (2018)

    Google Scholar 

  20. Jiao, S., Feng, J., Gao, Y., Lei, T., Yuan, X.: Visual cryptography in single-pixel imaging. arXiv preprint arXiv:1911.05033 (2019)

  21. Jin, M., Hirsch, M., Favaro, P.: Learning face deblurring fast and wide. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 745–753 (2018)

    Google Scholar 

  22. Khan, S.S., Adarsh, V., Boominathan, V., Tan, J., Veeraraghavan, A., Mitra, K.: Towards photorealistic reconstruction of highly multiplexed lensless images. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 7860–7869 (2019)

    Google Scholar 

  23. Levin, A., Fergus, R., Durand, F., Freeman, W.T.: Image and depth from a conventional camera with a coded aperture. ACM Trans. Graph. (TOG) 26(3), 70 (2007)

    Article  Google Scholar 

  24. Liang, C.K., Lin, T.H., Wong, B.Y., Liu, C., Chen, H.H.: Programmable aperture photography: multiplexed light field acquisition. ACM Trans. Graph. (TOG) 27, 55 (2008)

    Google Scholar 

  25. Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., Song, L.: Sphereface: Deep hypersphere embedding for face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 212–220 (2017)

    Google Scholar 

  26. Nagahara, H., Zhou, C., Watanabe, T., Ishiguro, H., Nayar, S.K.: Programmable aperture camera Using LCoS. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6316, pp. 337–350. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15567-3_25

    Chapter  Google Scholar 

  27. Nguyen Canh, T., Nagahara, H.: Deep compressive sensing for visual privacy protection in flatcam imaging. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (2019)

    Google Scholar 

  28. Nikonorov, A.V., Petrov, M., Bibikov, S.A., Kutikova, V.V., Morozov, A., Kazanskii, N.L.: Image restoration in diffractive optical systems using deep learning and deconvolution. Comput. Opt. 41(6), 875–887 (2017)

    Article  Google Scholar 

  29. Nodari, A., Vanetti, M., Gallo, I.: Digital privacy: replacing pedestrians from google street view images. In: Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), pp. 2889–2893. IEEE (2012)

    Google Scholar 

  30. Padilla-López, J.R., Chaaraoui, A.A., Flórez-Revuelta, F.: Visual privacy protection methods: a survey. Exp. Syst. Appl. 42(9), 4177–4195 (2015)

    Article  Google Scholar 

  31. Pittaluga, F., Koppal, S.J.: Privacy preserving optics for miniature vision sensors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 314–324 (2015)

    Google Scholar 

  32. Pittaluga, F., Koppal, S.J.: Pre-capture privacy for small vision sensors. IEEE Trans. Pattern Anal. Mach. Intell. 39(11), 2215–2226 (2016)

    Article  Google Scholar 

  33. Raskar, R.: Less is more: coded computational photography. In: Yagi, Y., Kang, S.B., Kweon, I.S., Zha, H. (eds.) ACCV 2007. LNCS, vol. 4843, pp. 1–12. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-76386-4_1

    Chapter  Google Scholar 

  34. Ren, D., Zhang, K., Wang, Q., Hu, Q., Zuo, W.: Neural blind deconvolution using deep priors. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

    Google Scholar 

  35. Ren, D., Zuo, W., Zhang, D., Xu, J., Zhang, L.: Partial deconvolution with inaccurate blur kernel. IEEE Trans. Image Process. 27(1), 511–524 (2017)

    Article  MathSciNet  Google Scholar 

  36. Ren, W., et al.: Deep non-blind deconvolution via generalized low-rank approximation. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 31, pp. 297–307. Curran Associates, Inc. (2018). http://papers.nips.cc/paper/7313-deep-non-blind-deconvolution-via-generalized-low-rank-approximation.pdf

  37. Schuler, C.J., Christopher Burger, H., Harmeling, S., Scholkopf, B.: A machine learning approach for non-blind image deconvolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1067–1074 (2013)

    Google Scholar 

  38. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 618–626 (2017)

    Google Scholar 

  39. Shen, Z., Lai, W.S., Xu, T., Kautz, J., Yang, M.H.: Deep semantic face deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8260–8269 (2018)

    Google Scholar 

  40. Sinha, A., Lee, J., Li, S., Barbastathis, G.: Lensless computational imaging through deep learning. Optica 4(9), 1117–1125 (2017)

    Article  Google Scholar 

  41. Sloane, N.J., Harwitt, M.: Hadamard transform optics (1979)

    Google Scholar 

  42. Son, H., Lee, S.: Fast non-blind deconvolution via regularized residual networks with long/short skip-connections. In: 2017 IEEE International Conference on Computational Photography (ICCP), pp. 1–10. IEEE (2017)

    Google Scholar 

  43. Tai, Y., Yang, J., Liu, X., Xu, C.: Memnet: a persistent memory network for image restoration. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4539–4547 (2017)

    Google Scholar 

  44. Thorpe, C., Li, F., Li, Z., Yu, Z., Saunders, D., Yu, J.: A coprime blur scheme for data security in video surveillance. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 3066–3072 (2013)

    Article  Google Scholar 

  45. Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., Tumblin, J.: Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing. ACM Trans. Graph. (TOG). 26, 69 (2007)

    Google Scholar 

  46. Wang, H., et al.: Cosface: large margin cosine loss for deep face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5265–5274 (2018)

    Google Scholar 

  47. Wang, R., Tao, D.: Training very deep CNNs for general non-blind deconvolution. IEEE Trans. Image Process. 27(6), 2897–2910 (2018)

    Article  MathSciNet  Google Scholar 

  48. Wang, Z.W., Vineet, V., Pittaluga, F., Sinha, S.N., Cossairt, O., Bing Kang, S.: Privacy-preserving action recognition using coded aperture videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)

    Google Scholar 

  49. Xu, L., Ren, J.S., Liu, C., Jia, J.: Deep convolutional neural network for image deconvolution. In: Advances in Neural Information Processing Systems, pp. 1790–1798 (2014)

    Google Scholar 

  50. Zhang, K., Xue, W., Zhang, L.: Non-blind image deconvolution using deep dual-pathway rectifier neural network. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2602–2606. IEEE (2017)

    Google Scholar 

  51. Zhang, L., Zuo, W.: Image restoration: from sparse and low-rank priors to deep priors [lecture notes]. IEEE Signal Process. Mag. 34(5), 172–179 (2017)

    Article  Google Scholar 

  52. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Panasonic Corporation, 1006 Kadoma, Kadoma, Osaka, Japan

    Yasunori Ishii & Satoshi Sato

  2. Chubu University, 1200 Matsumotocho, Kasugai, Aichi, Japan

    Takayoshi Yamashita

Authors
  1. Yasunori Ishii
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Satoshi Sato
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Takayoshi Yamashita
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yasunori Ishii .

Editor information

Editors and Affiliations

  1. University of Clermont Auvergne, Clermont Ferrand, France

    Adrien Bartoli

  2. Università degli Studi di Udine, Udine, Italy

    Andrea Fusiello

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ishii, Y., Sato, S., Yamashita, T. (2020). Privacy-Aware Face Recognition with Lensless Multi-pinhole Camera. In: Bartoli, A., Fusiello, A. (eds) Computer Vision – ECCV 2020 Workshops. ECCV 2020. Lecture Notes in Computer Science(), vol 12539. Springer, Cham. https://doi.org/10.1007/978-3-030-68238-5_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-68238-5_35

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-68237-8

  • Online ISBN: 978-3-030-68238-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation