Hongquan Li
ABSTRACT
Since the 20th century, the research on cipher neural network is more and more in-depth. Encryption scheme based on neural network is from layered homomorphic encryption scheme to fully homomorphic encryption scheme, the accuracy of neural network and the resource consumption in calculation are always the core of the research. This paper mainly studies the optimization of the excitation layer in the neural network. Since the convolution layer and pooling layer in the hidden layer are linear operations, they can support the secret state operation, and the excitation layer introduces nonlinear factors into the neural network in the form of nonlinear functions, so the secret state calculation cannot be directly applied to the layer. In order to solve this problem, this paper on the basis of CryptoNets model, only in the case of low degree to consider about the commonly used three kinds of activation function : Sigmoid, Tanh and ReLU function for the linear approximation, and proposed a new approximate method : An approximate function is constructed by using the structural characteristics of the derivative of the function, compared with the commonly used approximate method, a new method of approximate function replacement after activation function, the neural network prediction accuracy and operation time in both performance is more outstanding.
- Dowlin N, Gilad-Bachrach R, Laine K, CryptoNets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy[R].IEEE, 2016.Google Scholar
- F Bourse, Minelli M, Minihold M, Fast Homomorphic Evaluation of Deep Discretized Neural Networks[C].2017, 483-512.Google Scholar
- Hesamifard E, Takabi H, Ghasemi M .CryptoDL: Deep Neural Networks over Encrypted Data[R].2017.Google Scholar
- Badawi A A, Chao J, Jie L, The AlexNet Moment for Homomorphic Encryption: HCNN,the First Homomorphic CNN on Encrypted Data with GPUs[J].IEEE Transactions on Emerging Topics in Computing, 2020, PP(99):1-1Google Scholar
- Liu Xiaowen, Guo Dabo, Li Cong.An improvement of the activation function in convolutional neural networks [J].Test Technology, 2019,33 (02): 121-125Google Scholar
- Lai Ce.Analysis of the activation function in convolutional neural networks [J].Science and Technology Innovation, 2019,000 (033): 35-36Google Scholar
- Wang Shuangyin, Teng Guowen.Design of activation function optimization in convolutional neural networks [J].ICT, 2018,000 (001): 42-43Google Scholar
- Zhou Feiyan, Jin Linpeng, Dong Jun.Review of convolutional neural network studies [J].Journal of Computer Science, 2017,000 (6): 1-1Google Scholar
- Jiang Onbo, Wang Weiwei.Activation function optimization study [J].Sensor with Microsystems, 2018,02 (v.37;No.312):56-58Google Scholar
- Tian Juan, Li Yingxiang, Li Hongyan.Comparative study of the activation function in convolutional neural networks [J].Computer Systems Applications, 2018, v.27(07):45-51Google Scholar
- Qu Zhilin, Hu Xiaofei.Study on Convolutional Neural Network Based on improved activation functions [J].Computer Technology and Development, 2017,27 (012): 77-80Google Scholar
Recommendations
Wheat crop yield prediction using new activation functions in neural network
AbstractThis research mainly based on multilayer perceptron (MLP) neural networks technique of data mining to forecast the wheat crop yield at the district level. There are many statistical and simulation models available, but the proposed algorithm with ...
Modified Neural Network Activation Function
ICAIET '14: Proceedings of the 2014 4th International Conference on Artificial Intelligence with Applications in Engineering and TechnologyNeural Network is said to emulate the brain, though, its processing is not quite how the biological brain really works. The Neural Network has witnessed significant improvement since 1943 to date. However, modifications on the Neural Network mainly ...
Research on Gaussian-wavelet-type Activation Function of Neural Network Hidden Layer Based on Monte Carlo Method
RSVT '19: Proceedings of the 2019 International Conference on Robotics Systems and Vehicle TechnologyArtificial neural networks have developed rapidly in recent years and have been applied in the fields of image recognition, natural language processing, and pattern recognition. The activation function, as an integral part of the neural network, plays a ...
Comments