Skip to main content
SpringerLink
Log in

A privacy-preserving image retrieval method based on deep learning and adaptive weighted fusion

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

With the development of big data and cloud computing, more and more data owners store the data in cloud server. Considering privacy preserving, image data need to be encrypted before uploaded to the cloud, which will lead to inefficient image retrieval of ciphertext domain. Therefore, the challenge of encrypted image retrieval is how to improve the performance. Toward this goal, this paper proposes a privacy-preserving image retrieval method based on deep learning and adaptive weighted fusion. Firstly, extracting low-level feature EHD (edge histogram descriptor), BOW (bag of words) and high-level semantic feature of images. Secondly, reducing the dimension of 1024-dim high-level semantic feature by PCA (principal component analysis), and the three features were binarized. Then these types of features are adaptively fused. Finally, constructing a prefilter table for fusion features to improve search efficiency by locality sensitive hashing (LSH) algorithm. K-nearest neighbor (KNN) algorithm and logistic encryption method were used to protect the privacy of fused features and images, respectively. The experiments show that the proposed method can not only ensure image security but also improve the retrieval accuracy of encrypted image.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Xia, Z.H., Zhang, L.G., Liu, D.D.: Attribute-based access control scheme with efficient revocation in cloud computing. China Commun. 13(7), 92–99 (2016)

    Article  Google Scholar 

  2. Qi, L.Y., Meng, S.M., Zhang, X.Y., Wang, R.L., Xu, X.L., Zhou, Z.L., Dou, W.H.: An exception handling approach for privacy-preserving service recommendation failure in a cloud environment. Sensors 18(7), 1–11 (2018)

    Article  Google Scholar 

  3. Qi, L.Y., Yu, J.G., Zhou, Z.L.: An invocation cost optimization method for web services in cloud environment. Sci. Program. 2017, 4358536–4358544 (2017)

    Google Scholar 

  4. Xia, Z.H., Zhu, Y., Sun, X.M., Chen, L.: Secure semantic expansion based search over encrypted cloud data supporting similarity ranking. J. Cloud Comput. 3(1), 1–11 (2014)

    Article  Google Scholar 

  5. Fu, Z., Wu, X., Guan, C., Sun, X.M., Ren, K.: Toward efficient multi-keyword fuzzy search over encrypted outsourced data with accuracy improvement. IEEE Trans. Inf. Forensics Secur. 11(12), 2706–2716 (2016)

    Article  Google Scholar 

  6. Lu, W., Varna, A. L., Swaminathan, A., Wu, M.: Secure image retrieval through feature protection. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, pp. 1533–1536 (2009)

  7. Hsu, C.Y., Lu, C.S., Pei, S.C.: Image feature extraction in encrypted domain with privacy-preserving SIFT. IEEE Trans. Image Process. 21(11), 4593–4607 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bai, Y., Zhuo, L., Cheng, B., Peng, Y. F.: Surf feature Extraction in encrypted domain. In: 2014 IEEE international conference on multimedia and expo, pp. 1–6 (2014)

  9. Lu, W., Varna, A.L., Wu, M.: Confidentiality-preserving image search: a comparative study between homomorphic encryption and distance-preserving randomization. IEEE Access. 2(1), 125–141 (2014)

    Google Scholar 

  10. Cheng, H., Zhang, X., Yu, J., Li, F.: Markov process-based retrieval for encrypted jpeg images. Eurasip J. Inf. Security. 2016(1), 1–9 (2016)

    Article  Google Scholar 

  11. Xia, Z.H., Zhu, Y., Sun, X.M., Qin, Z., Ren, K.: Towards privacy-preserving content-based image retrieval in cloud computing. IEEE Trans. Cloud Comput. 6(1), 276–286 (2018)

    Article  Google Scholar 

  12. Xia, Z.H., Xiong, N., Vasilakos, V., Sun, X.M.: EPCBIR: an efficient and privacy-preserving content-based image retrieval scheme in cloud computing. Inf. Sci. 387, 195–204 (2017)

    Article  Google Scholar 

  13. Qin, J.H., Li, H., Xiang, X.Y., Tan, Y., Pan, W.Y., Xiong, N.N.: An encrypted image retrieval method based on Harris Corner optimization and LSH in cloud computing. IEEE Access. 7(1), 24626–24633 (2018)

    Google Scholar 

  14. Qi, L.Y., Zhang, X.Y., Dou, W.C., Hu, C.H., Yang, C., Chen, Jinjun: A two-stage locality-sensitive hashing based approach for privacy-preserving mobile service recommendation in Cross-platform edge environment. Future Gener. Comput. Syst. 88, 636–643 (2018)

    Article  Google Scholar 

  15. Xu, Y.W., Qi, L.Y., Dou, W.C., Yu, J.G.: Privacy-preserving and scalable service recommendation based on SimHash in a distributed cloud environment. Complexity 2017, 3437854–3437862 (2017)

    Google Scholar 

  16. Xiang, L.Y., Shen, X.B., Qin, J.H., Hao, W.: Discrete multi-graph hashing for large-scale visual search. Neural Process. Lett. 49(3), 1055–1069 (2019)

    Article  Google Scholar 

  17. Li, H., Qin, J.H., Xiang, X.Y., Pan, L.L., Ma, W.T., Xiong, N.N.: An efficient image matching algorithm based on adaptive threshold and RANSAC. IEEE Access. 6(1), 66963–66971 (2018)

    Article  Google Scholar 

  18. Liu, X.W., Wang, L., Zhang, J., Yin, J.P., Liu, H.: Global and local structure preservation for feature selection. IEEE Trans. Neural Netw. Learn. Syst. 25(6), 1083–1095 (2014)

    Article  Google Scholar 

  19. Ma, W.T., Qin, J.H., Xiang, X.Y., Tan, Y., Luo, Y.J., Xiong, N.N.: Adaptive median filtering algorithm based on divide and conquer and its application in CAPTCHA recognition. Comput. Mater. Continua. 58(3), 665–677 (2019)

    Article  Google Scholar 

  20. Liu, Z., Blasch, E., John, V.: Statistical comparison of image fusion algorithms: recommendations. Inf. Fusion. 36, 251–260 (2017)

    Article  Google Scholar 

  21. Xie, B., Qin, J.H., Xiang, X.Y., Li, H., Pan, L.L.: An image retrieval algorithm based on GIST and SIFT features. Int. J. Netw. Secur. 20(4), 609–616 (2018)

    Google Scholar 

  22. Qin, J.H., Sun, X.M., Xiang, X.Y., Niu, C.M.: Principal feature selection and fusion method for image steganalysis. J. Electron. Imaging. 18(3), 1–14 (2009)

    Article  Google Scholar 

  23. Yu, S., Niu, D., Zhao, X., Liu, M.: Color image retrieval based on the hypergraph and the fusion of two descriptors. In: Proceedings of the 10th international congress on image and signal processing, BioMedical engineering and informatics, pp. 1–6 (2017)

  24. Zheng, L., Wang, S., Tian, L., He, F., Liu, Z., Tian, Q.: Query-adaptive late fusion for image search and person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1741–1750 (2015)

  25. Zhu, X., Jing, X.Y., Wu, F., Wang, Y., Zuo, W., Zheng, W.S.: Learning heterogeneous dictionary pair with feature projection matrix for pedestrian video retrieval via single query image. In: Proceedings of the thirty-first AAAI conference on artificial intelligence, pp. 4341–4348 (2017)

  26. Zhang, J.M., Wu, Y., Feng, W.J., Wang, J.: Spatially attentive visual tracking using multi-model adaptive response fusion. IEEE Access. 7(1), 83873–83887 (2019)

    Article  Google Scholar 

  27. Pan, L.L., Qin, J.H., Chen, H., Xiang, X.Y., Li, C., Chen, R.: Image augmentation-based food recognition with convolutional neural networks. Comput. Mater. Continua. 59(1), 297–313 (2019)

    Article  Google Scholar 

  28. Pan, W.Y., Qin, J.H., Xiang, X.Y., Wu, Y., Tan, Y., Xiang, L.Y.: A smart mobile diagnosis system for citrus diseases based on densely connected convolutional networks. IEEE Access. 7, 87534–87542 (2019)

    Article  Google Scholar 

  29. Zhang, J.M., Lu, C.Q., Li, X.D., Kim, H.J., Wang, J.: A full convolutional network based on DenseNet for remote sensing scene classification. Math. Biosci. Eng. 16(5), 3345–3367 (2019)

    Article  Google Scholar 

  30. Wang, J., Qin, J.H., Xiang, X.Y., Tan, Y., Pan, N.: CAPTCHA recognition based on deep convolutional neural network. Math. Biosci. Eng. 16(5), 5851–5861 (2019)

    Article  MathSciNet  Google Scholar 

  31. Zhang, J.M., Jin, X.K., Sun, J., Wang, J., Sangaiah, A.K.: Spatial and semantic convolutional features for robust visual object tracking. Multimedia Tools Appl. (2018). https://doi.org/10.1007/s11042-018-6562-8

    Article  Google Scholar 

  32. Zhang, J.M., Lu, C.Q., Wang, J., Wang, L., Yue, X.G.: Concrete cracks detection based on FCN with dilated convolution. Appl. Sci. 9(13), 2686 (2019)

    Article  Google Scholar 

  33. Zhang, J.M., Jin, X.K., Sun, J., Wang, J., Li, K.Q.: Dual model learning combined with multiple feature selection for accurate visual tracking. IEEE Access. 7(1), 43956–43969 (2019)

    Article  Google Scholar 

  34. Datar, M., Nicole, I., Piotr, I., Vahab, S.M.: Locality-sensitive hashing scheme based on p-stable distributions. In: Proceedings of the twentieth annual symposium on Computational geometry, pp. 253–262 (2004)

  35. Delfs, H., Knebl, H.: Introduction to cryptography: principles and applications. Inf. Secur. Cryptogr. Texts Monogr. 4(4), 206–207 (2015)

    MATH  Google Scholar 

  36. Wang, J.Z., Li, J., Wiederhold, G.: Simplicity: semantics-sensitive integrated matching for picture libraries. IEEE Trans. Pattern Anal. Mach. Intell. 23(9), 947–963 (2007)

    Article  Google Scholar 

  37. Zhou, Z.L., Wu, J.N., Sun, X.M.: Multiple distances-based coding: toward scalable feature matching for large-scale web image search. IEEE Trans. Big Data. (2019). https://doi.org/10.1109/TBDATA.2019.2919570

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (no. 61772561), the Key Research & Development Plan of Hunan Province (nos. 2018NK2012, 2019SK2022), the Science Research Projects of Hunan Provincial Education Department (nos. 18A174 and 18C0262), and the Science & Technology Innovation Platform and Talent Plan of Hunan Province (2017TP1022).

Author information

Authors and Affiliations

  1. College of Computer Science and Information Technology, Central South University of Forestry and Technology, Changsha, China

    Jiaohua Qin, Jianhua Chen, Xuyu Xiang, Yun Tan, Wentao Ma & Jing Wang

Authors
  1. Jiaohua Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianhua Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xuyu Xiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yun Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wentao Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiaohua Qin.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qin, J., Chen, J., Xiang, X. et al. A privacy-preserving image retrieval method based on deep learning and adaptive weighted fusion. J Real-Time Image Proc 17, 161–173 (2020). https://doi.org/10.1007/s11554-019-00909-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-019-00909-3

Keywords

Search

Navigation