Abstract
Distributed big data computing environments such as machine learning are widely deployed and applied on the cloud. However, since cloud servers can easily access user data, it leads to serious data leakage problems. As a potential technology, Fully Homomorphic Encryption (FHE) is often used in the field of privacy-preserving machine learning. However, in order to reduce the multiplicative depth of FHE, the neural network prunes the number of network layers, resulting in low inference accuracy. A learning model named Broad Learning System (BLS) has the characteristics of shallow model depth and low complexity. Based on this mode, we propose a privacy-preserving inference algorithm with low multiplicative depth, namely broad learning inference based on fully homomorphic encryption. We also extend the algorithm to the BLS model with incremental learning. We implement the privacy-preserving BLS for the first time using TENSEAL’s CKKS scheme, and also verify the effectiveness of BLS inference with incremental learning. Experimental evaluations demonstrate the inference accuracy of 0.928 and 0.672 for datasets of MNIST and NORB, respectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Al Badawi, A., et al.: Towards the AlexNet moment for homomorphic encryption: HCNN, the first homomorphic CNN on encrypted data with GPUs. arXiv e-prints pp. arXiv-1811 (2018)
Bajard, J.-C., Eynard, J., Hasan, M.A., Zucca, V.: A full RNS variant of FV like somewhat homomorphic encryption schemes. In: Avanzi, R., Heys, H. (eds.) SAC 2016. LNCS, vol. 10532, pp. 423–442. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69453-5_23
Bost, R., Popa, R.A., Tu, S., Goldwasser, S.: Machine learning classification over encrypted data. Cryptology ePrint Archive (2014)
Brakerski, Z.: Fully homomorphic encryption without modulus switching from classical GapSVP. In: Safavi-Naini, R., Canetti, R. (eds.) CRYPTO 2012. LNCS, vol. 7417, pp. 868–886. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32009-5_50
Brakerski, Z., Gentry, C., Vaikuntanathan, V.: (Leveled) fully homomorphic encryption without bootstrapping. ACM Trans. Comput. Theory 6(3), 1–36 (2014)
Brutzkus, A., Gilad-Bachrach, R., Elisha, O.: Low latency privacy preserving inference. In: International Conference on Machine Learning, pp. 812–821. PMLR (2019)
Chabanne, H., De Wargny, A., Milgram, J., Morel, C., Prouff, E.: Privacy-preserving classification on deep neural network. Cryptology ePrint Archive (2017)
Chen, C.P., Liu, Z.: Broad learning system: an effective and efficient incremental learning system without the need for deep architecture. IEEE Trans. Neural Netw. Learn. Syst. 29(1), 10–24 (2017)
Cheon, J.H., Han, K., Kim, A., Kim, M., Song, Y.: A full RNS variant of approximate homomorphic encryption. In: Cid, C., Jacobson, Jr., M. (eds.) Selected Areas in Cryptography (SAC 2018). LNCS, vol. 11349, pp. 347–368. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-10970-7_16
Cheon, J.H., Kim, A., Kim, M., Song, Y.: Homomorphic encryption for arithmetic of approximate numbers. In: Takagi, T., Peyrin, T. (eds.) ASIACRYPT 2017. LNCS, vol. 10624, pp. 409–437. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-70694-8_15
Fan, J., Vercauteren, F.: Somewhat practical fully homomorphic encryption. Cryptology ePrint Archive (2012)
Gentry, C.: A fully homomorphic encryption scheme. Stanford University (2009)
Gilad-Bachrach, R., Dowlin, N., Laine, K., Lauter, K., Naehrig, M., Wernsing, J.: CryptoNets: applying neural networks to encrypted data with high throughput and accuracy. In: International Conference on Machine Learning, pp. 201–210. PMLR (2016)
Hesamifard, E., Takabi, H., Ghasemi, M.: CryptoDL: deep neural networks over encrypted data. arXiv preprint arXiv:1711.05189 (2017)
Hoerl, A.E., Kennard, R.W.: Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1), 55–67 (1970)
Ishiyama, T., Suzuki, T., Yamana, H.: Highly accurate CNN inference using approximate activation functions over homomorphic encryption. In: 2020 IEEE International Conference on Big Data (Big Data), pp. 3989–3995. IEEE (2020)
Rivest, R.L., Adleman, L., Dertouzos, M.L., et al.: On data banks and privacy homomorphisms. Found. Secure Comput. 4(11), 169–180 (1978)
Yuan, X., Chen, J., Zhang, N., Fang, X., Liu, D.: A federated bidirectional connection broad learning scheme for secure data sharing in internet of vehicles. China Commun. 18(7), 117–133 (2021)
Acknowledgement
This work was supported by the Key-Area Research and Development Program of Guangdong Province (No. 2020B010164003).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Deng, X., Sang, Y., Li, Z. (2023). Broad Learning Inference Based on Fully Homomorphic Encryption. In: Takizawa, H., Shen, H., Hanawa, T., Hyuk Park, J., Tian, H., Egawa, R. (eds) Parallel and Distributed Computing, Applications and Technologies. PDCAT 2022. Lecture Notes in Computer Science, vol 13798. Springer, Cham. https://doi.org/10.1007/978-3-031-29927-8_38
Download citation
DOI: https://doi.org/10.1007/978-3-031-29927-8_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-29926-1
Online ISBN: 978-3-031-29927-8
eBook Packages: Computer ScienceComputer Science (R0)