Skip to main content
Springer Nature Link
Log in

Towards Real-Time Hidden Speaker Recognition by Means of Fully Homomorphic Encryption

  • Conference paper
  • First Online:
Information and Communications Security (ICICS 2020)

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 12282))

Included in the following conference series:

Abstract

Securing Neural Network (NN) computations through the use of Fully Homomorphic Encryption (FHE) is the subject of a growing interest in both communities. Among different possible approaches to that topic, our work focuses on applying FHE to hide the model of a neural network-based system in the case of a plain input. In this paper, using the TFHE homomorphic encryption scheme, we propose an efficient method for an \(\mathsf {argmin}\) computation on an arbitrary number of encrypted inputs and an asymptotically faster - though levelled - equivalent scheme. Using these schemes and a unifying framework for LWE-based homomorphic encryption schemes (Chimera), we implement a practically efficient, homomorphic speaker recognition system using the embedding-based neural net system VGGVox. This work can be applied to all other similar Euclidean embedding-based recognition systems (e.g. Google’s FaceNet). While maintaining the best-in-class classification rate of the VGGVox system, we demonstrate a speaker-recognition system that can classify a speech sample as coming from one out of 50 hidden speaker models in less than one minute.

S. Carpov—This work was done in part while this author was at CEA, LIST.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    The way these samples are selected may depend on some meta-data and is beyond the scope of this paper.

  2. 2.

    This bootstrapping is only a slight variation on the bootstrapping procedure introduced in [13], we just add a public rotation to the bootstrap operation used in [5].

  3. 3.

    We implicitly write the possible values of \(\mu \) and the output value \(\mu _0\) as members of the torus space \(\mathbb {T}\). Alternatively, we also refer to 1/b as the value 1. This is arbitrary but allows us to represent the bootstrap operation very intuitively in Fig. 2.

  4. 4.

    In the case where \(\mathbb {d}_k = \mathbb {d}_l\) the sign bootstrap yields a random output. This is actually also the case when \(\mathbb {d}_k\) is “close” to \(\mathbb {d}_l\): this means the difference is in the red zone around 0 seen in Fig. 2. See Sect. 5.1 for implications.

  5. 5.

    This greatly reduces the stress on the parameters induced from the \(\mathsf {MUX}\) gate.

  6. 6.

    This time where \(b = 4\) and where the 0 and the 1 outputs are swapped.

  7. 7.

    https://bitbucket.org/malb/lwe-estimator/raw/HEAD/estimator.py.

References

  1. Albrecht, M., Player, R., Scott, S.: On the concrete hardness of learning with errors. J. Math. Cryptol., ePrint Archive 2015/046 (2015)

    Google Scholar 

  2. Ball, M., Carmer, B., Malkin, T., Rosulek, M., Schimanski, N.: Garbled neural networks are practical. Cryptology ePrint Archive, Report 2019/338 (2019)

    Google Scholar 

  3. Bergamaschi, F., Halevi, S., Halevi, T.T., Hunt, H.: Homomorphic training of 30,000 logistic regression models. In: Deng, R.H., Gauthier-Umaña, V., Ochoa, M., Yung, M. (eds.) ACNS 2019. LNCS, vol. 11464, pp. 592–611. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-21568-2_29

    Chapter  Google Scholar 

  4. Boura, C., Gama, N., Georgieva, M.: Chimera: a unified framework for B/FV, TFHE and HEAAN fully homomorphic encryption and predictions for deep learning. Cryptology ePrint Archive, Report 2018/758 (2018)

    Google Scholar 

  5. Bourse, F., Minelli, M., Minihold, M., Paillier, P.: Fast homomorphic evaluation of deep discretized neural networks. In: Shacham, H., Boldyreva, A. (eds.) CRYPTO 2018. LNCS, vol. 10993, pp. 483–512. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-96878-0_17

    Chapter  Google Scholar 

  6. Brakerski, Z., Gentry, C., Vaikuntanathan, V.: Fully homomorphic encryption without bootstrapping. Cryptology ePrint Archive, Report 2011/277 (2011)

    Google Scholar 

  7. Carpov, S., Gama, N., Georgieva, M., Troncoso-Pastoriza, J.R.: Privacy-preserving semi-parallel logistic regression training with fully homomorphic encryption. Cryptology ePrint Archive, Report 2019/101 (2019)

    Google Scholar 

  8. Chabanne, H., Lescuyer, R., Milgram, J., Morel, C., Prouff, E.: Recognition Over Encrypted Faces. In: Renault, É., Boumerdassi, S., Bouzefrane, S. (eds.) MSPN 2018. LNCS, vol. 11005. Springer, Heidelberg (2019). https://doi.org/10.1007/978-3-030-03101-5_16

    Chapter  Google Scholar 

  9. Chabanne, H., de Wargny, A., Milgram, J., Morel, C., Prouff, E.: Privacy-preserving classification on deep neural network. Cryptology ePrint Archive, Report 2017/035 (2017)

    Google Scholar 

  10. Chen, H., Chillotti, I., Dong, Y., Poburinnaya, O., Razenshteyn, I., Riazi, M.S.: SANNS: scaling up secure approximate k-nearest neighbors search. Cryptology ePrint Archive, Report 2019/359 (2019)

    Google Scholar 

  11. Chen, H., Laine, K., Player, R.: Simple encrypted arithmetic library - seal v2.1 (2017)

    Google Scholar 

  12. Cheon, J.H., Kim, D., Park, J.H.: Towards a practical clustering analysis over encrypted data. IACR Cryptology ePrint Archive (2019)

    Google Scholar 

  13. Chillotti, I., Gama, N., Georgieva, M., Izabachène, M.: Faster fully homomorphic encryption: Bootstrapping in less than 0.1 seconds. Cryptology ePrint Archive, Report 2016/870 (2016)

    Google Scholar 

  14. Chillotti, I., Gama, N., Georgieva, M., Izabachène, M.: Improving TFHE: faster packed homomorphic operations and efficient circuit bootstrapping. IACR Cryptology ePrint Archive, p. 430 (2017)

    Google Scholar 

  15. Chillotti, I., Gama, N., Georgieva, M., Izabachène, M.: TFHE: fast fully homomorphic encryption library, August 2016. https://tfhe.github.io/tfhe/

  16. Chung, J.S., Nagrani, A., Zisserman, A.: VoxCeleb2: deep speaker recognition. CoRR (2018)

    Google Scholar 

  17. Demmler, D., Schneider, T., Zohner, M.: ABY - a framework for efficient mixed-protocol secure two-party computation (2015)

    Google Scholar 

  18. Erkin, Z., Franz, M., Guajardo, J., Katzenbeisser, S., Lagendijk, I., Toft, T.: Privacy-preserving face recognition. In: Goldberg, I., Atallah, M.J. (eds.) PETS 2009. LNCS, vol. 5672, pp. 235–253. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03168-7_14

    Chapter  Google Scholar 

  19. Failla, P., Barni, M., Catalano, D., di Raimondo, M., Labati, R., Bianchi, T.: Privacy- preserving fingercode authentication (2010)

    Google Scholar 

  20. Fan, J., Vercauteren, F.: Somewhat practical fully homomorphic encryption. Cryptology ePrint Archive, Report 2012/144 (2012)

    Google Scholar 

  21. Gentry, C.: Fully homomorphic encryption using ideal lattices. In: Proceedings of the Forty-first Annual ACM Symposium on Theory of Computing, STOC 2009 (2009)

    Google Scholar 

  22. Gentry, C., Sahai, A., Waters, B.: Homomorphic encryption from learning with errors: conceptually-simpler, asymptotically-faster, attribute-based. Cryptology ePrint Archive, Report 2013/340 (2013)

    Google Scholar 

  23. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)

    MATH  Google Scholar 

  24. Halevi, S., Shoup, V.: Algorithms in HElib. In: Garay, J.A., Gennaro, R. (eds.) CRYPTO 2014. LNCS, vol. 8616, pp. 554–571. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44371-2_31

    Chapter  MATH  Google Scholar 

  25. Huang, Y., Malka, L., Evans, D., Katz, J.: Efficient privacy-preserving biometric identification. In: NDSS (2011)

    Google Scholar 

  26. Izabachène, M., Sirdey, R., Zuber, M.: Practical fully homomorphic encryption for fully masked neural networks. In: Mu, Y., Deng, R.H., Huang, X. (eds.) CANS 2019. LNCS, vol. 11829, pp. 24–36. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-31578-8_2

    Chapter  Google Scholar 

  27. Jäschke, A., Armknecht, F.: Unsupervised machine learning on encrypted data. IACR Cryptology ePrint Archive (2018)

    Google Scholar 

  28. Kim, M., Song, Y., Wang, S., Xia, Y., Jiang, X.: Secure logistic regression based on homomorphic encryption: design and evaluation. JMIR Med. Inf. 6, e19 (2018)

    Article  Google Scholar 

  29. Lyubashevsky, V., Peikert, C., Regev, O.: On ideal lattices and learning with errors over rings. In: Gilbert, H. (ed.) EUROCRYPT 2010. LNCS, vol. 6110, pp. 1–23. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13190-5_1

    Chapter  Google Scholar 

  30. Nagrani, A., Chung, J.S., Zisserman, A.: VoxCeleb: a large-scale speaker identification dataset (2017)

    Google Scholar 

  31. Nandakumar, K., Ratha, N.K., Pankanti, S., Halevi, S.: Towards deep neural network training on encrypted data. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 40–48 (2019)

    Google Scholar 

  32. Regev, O.: On lattices, learning with errors, random linear codes, and cryptography. In: Proceedings of the 37th Annual ACM Symposium on Theory of Computing. ACM (2005)

    Google Scholar 

  33. Rivest, R.L., Adleman, L., Dertouzos, M.L.: On data banks and privacy homomorphisms. In: Foundations of Secure Computation, pp. 169–179. Academia Press (1978)

    Google Scholar 

  34. Rouhani, B.D., Riazi, M.S., Koushanfar, F.: DeepSecure: scalable provably-secure deep learning. CoRR (2017)

    Google Scholar 

  35. Sadeghi, A.-R., Schneider, T., Wehrenberg, I.: Efficient privacy-preserving face recognition. In: Lee, D., Hong, S. (eds.) ICISC 2009. LNCS, vol. 5984, pp. 229–244. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14423-3_16

    Chapter  Google Scholar 

  36. Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. CoRR (2015)

    Google Scholar 

  37. Microsoft SEAL (release 3.2). Microsoft Research, Redmond, WA, February 2019. https://github.com/Microsoft/SEAL

  38. Shaul, H., Feldman, D., Rus, D.: Scalable secure computation of statistical functions with applications to k-nearest neighbors. CoRR (2018)

    Google Scholar 

  39. Xie, P., Bilenko, M., Finley, T., Gilad-Bachrach, R., Lauter, K.E., Naehrig, M.: Crypto-nets: neural networks over encrypted data. CoRR (2014)

    Google Scholar 

  40. Yao, A.C.C.: How to generate and exchange secrets. In: Proceedings of the 27th Annual Symposium on Foundations of Computer Science, SFCS 1986, IEEE Computer Society (1986)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CEA, LIST, Palaiseau, France

    Martin Zuber & Renaud Sirdey

  2. Inpher, Lausanne, Switzerland

    Sergiu Carpov

Authors
  1. Martin Zuber
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sergiu Carpov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Renaud Sirdey
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martin Zuber .

Editor information

Editors and Affiliations

  1. Technical University of Denmark, Kongens Lyngby, Denmark

    Weizhi Meng

  2. Hamburg University of Technology, Hamburg, Germany

    Dieter Gollmann

  3. Technical University of Denmark, Kongens Lyngby, Denmark

    Christian D. Jensen

  4. Singapore University of Technology and Design, Singapore, Singapore

    Jianying Zhou

A Parameters

A Parameters

In this appendix we present the security parameters we use for both FV and TFHE. This corresponds to Tables 1, 2 and 3. The security of chosen parameters (as an example for the first parameters from Table 1) was asserted using the following scripts (lwe-estimator commit a276755):

  • FV:

    figure a

  • TFHE:

    figure b
Table 1. Tables presenting the security parameters for both FV and TFHE. In the left table, the security parameters for FV. In the right table, the security parameters for TFHE (\(q=1\)). \(\sigma \) is the standard deviation of the Gaussian noise. Because we use both FV and TFHE, we need to choose the tighter security constraint between the two. The actual standard deviations chosen in the case of \(N=1024\) for both 80 and 110 security levels are therefore written in green in the tables. The ones we cannot chose are in red.
Full size table
Table 2. The parameters set in the SEAL library for the initial distance computations. N is the size of the initial ciphertext polynomials. The value of p needs to be high enough to prevent the distance values from “overflowing” and needs to verify \(q = 1 \mod p\) in order to reduce noise propagation.
Full size table
Table 3. Tables presenting the appropriate parameters for both the matrix and tree schemes. In the upper table, the parameter set in the case of the “matrix” \(\mathsf {argmin}\) computation. In the lower table, the parameter set in the case of the “tree” scheme. \(N_b\) (resp. \(N_s\)) is the size of the bootstrapping key (resp. the key-switching key) polynomials and \(\sigma _b\) (resp. \(\sigma _s\)) the standard deviation of the Gaussian noise in the bootstrapping key (resp. the key-switching key).
Full size table

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zuber, M., Carpov, S., Sirdey, R. (2020). Towards Real-Time Hidden Speaker Recognition by Means of Fully Homomorphic Encryption. In: Meng, W., Gollmann, D., Jensen, C.D., Zhou, J. (eds) Information and Communications Security. ICICS 2020. Lecture Notes in Computer Science(), vol 12282. Springer, Cham. https://doi.org/10.1007/978-3-030-61078-4_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-61078-4_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-61077-7

  • Online ISBN: 978-3-030-61078-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics