Abstract
The image classification models based on neural networks recently have outperformed most of the traditional models, and rapidly been developed and implemented by industry because of the capability of qualifying various computer vision tasks. Hence, the exposure of users’ image data to unauthorized powerful models causes more information leak in a shorter time. Through experiments, we find that for each input image, the change of the image’s prediction scores by each pixels’ RGB value change is different. Also, the pattern of the sensitivity on each pixel is highly related to the category and composition of the input image. By utilizing this feature, we present Pivot Pixel Noise Generator by Particle Swarm Optimization to generate noise points on original images to lower the target model’s accuracy of correctly predicting the target image’s label, so to protect the information contained in the target image from the image classification models. The model performs in a semi-black-box manner and balances the number of queries to the target and total number of modified points. We also propose an initialization strategy for the model, PSO Knowledge Transfer, which initializes the model’s parameters with experience learned from previous runs to further reduce the number of query times and noise points. The model is evaluated using the image classification benchmark model ResNet50 and shows an advantage compared to the baseline algorithm.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
He, Z., Cai, Z., Yu, J.: Latent-data privacy preserving with customized data utility for social network data. IEEE Trans. Veh. Technol. 67(1), 665–673 (2018)
Çiftçi, S., Akyüz, A.O., Ebrahimi, T.: A reliable and reversible image privacy protection based on false colors. IEEE Trans. Multimed. 20(1), 68–81 (2018)
Cai, Z., He, Z.: Trading private range counting over big IoT data. In: 2019 IEEE 39th ICDCS, pp. 144–153, July 2019
Zhao, Y., Yu, Y., Li, Y., Han, G., Du, X.: Machine learning based privacy-preserving fair data trading in big data market. Inf. Sci. 478, 449–460 (2019)
Gao, W., Yu, W., Liang, F., Hatcher, W.G., Lu, C.: Privacy-preserving auction for big data trading using homomorphic encryption. IEEE Trans. Netw. Sci. Eng., 1 (2018)
Abadi, M., et al.: Deep learning with differential privacy. In: Weippl, E.R., Katzenbeisser, S., Kruegel, C., Myers, A.C., Halevi, S. (eds.) ACM CCS, pp. 308–318. ACM (2016)
Shokri, R., Shmatikov, V.: Privacy-preserving deep learning. In: Ray, I., Li, N., Kruegel, C. (eds.) ACM CCS, pp. 1310–1321. ACM (2015)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25. Curran Associates, Inc., pp. 1097–1105 (2012)
Zheng, X., Cai, Z., Li, Y.: Data linkage in smart internet of things systems: a consideration from a privacy perspective. IEEE Commun. Mag. 56(9), 55–61 (2018)
https://aws.amazon.com/sagemaker/. Accessed 31 Jan 2020
https://cloud.google.com/mlengine/docs/technical-overview/. Accessed 31 Jan 2020
https://azure.microsoft.com/enus/services/machine-learning-studio/. Accessed 31 Jan 2020
Song, C., Ristenpart, T., Shmatikov, V.: Machine learning models that remember too much. CoRR abs/1709.07886 (2017)
Su, J., Vargas, D.V., Sakurai, K.: One pixel attack for fooling deep neural networks. IEEE Trans. Evol. Comput. 23(5), 828–841 (2019)
Kennedy, J., Eberhart, R.C.: Particle swarm optimization. In: Proceedings of IEEE IJCNN, Washington, DC, USA, vol. 4, pp. 1942–1948. IEEE Computer Society, November 1995
Szegedy, C., et al.: Intriguing properties of neural networks. In: Bengio, Y., LeCun, Y. (eds.) ICLR (Poster) (2014)
Nguyen, A.M., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: high confidence predictions for unrecognizable images. In: CVPR, pp. 427–436 IEEE Computer Society (2015)
Papernot, N., et al.: The limitations of deep learning in adversarial settings. In: EuroS&P, pp. 372–387. IEEE (2016)
Papernot, N., McDaniel, P.D., Goodfellow, I.J., Jha, S., Celik, Z.B., Swami, A.: Practical black-box attacks against machine learning. In: Karri, R., Sinanoglu, O., Sadeghi, A.R., Yi, X. (eds.) AsiaCCS, pp. 506–519. ACM (2017)
LeCun, Y., Cortes, C.: MNIST handwritten digit database (2010)
Stallkamp, J., Schlipsing, M., Salmen, J., Igel, C.: Man vs. computer: benchmarking machine learning algorithms for traffic sign recognition. Neural Netw. 32, 323–332 (2012)
Krizhevsky, A.: Learning multiple layers of features from tiny images, pp. 32–33 (2009)
Zhang, Q., Wang, K., Zhang, W., Hu, J.: Attacking black-box image classifiers with particle swarm optimization. IEEE Access 7, 158051–158063 (2019)
Mosli, R., Wright, M., Yuan, B., Pan, Y.: They might not be giants: crafting black-box adversarial examples with fewer queries using particle swarm optimization. CoRR abs/1909.07490 (2019)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR abs/1512.03385 (2015)
Deng, J., Socher, R., Fei-Fei, L., Dong, W., Li, K., Li, L.J.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE CVPR, vol. 00, pp. 248–255, June 2009
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)
Acknowledgement
This research is supported, in part, by the SunTrust Fellowship Grant (ST20-07).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, J., Huang, Y., Pang, J., Xie, Z., Li, W. (2020). Image Privacy Protection by Particle Swarm Optimization Based Pivot Pixel Modification. In: Song, W., Lee, K., Yan, Z., Zhang, LJ., Chen, H. (eds) Internet of Things - ICIOT 2020. ICIOT 2020. Lecture Notes in Computer Science(), vol 12405. Springer, Cham. https://doi.org/10.1007/978-3-030-59615-6_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-59615-6_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59614-9
Online ISBN: 978-3-030-59615-6
eBook Packages: Computer ScienceComputer Science (R0)