Skip to main content
Springer Nature Link
Log in

Real-time image carrier generation based on generative adversarial network and fast object detection

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Image steganography aims to conceal the secret information inside another carrier image. And by embedding the information into the carrier image, the carrier image may suffer certain image distortion. Thus, not only the hiding algorithm should be carefully designed, but also the carrier image should be meticulously selected during the hiding process. This paper follows the idea of creating suitable cover images instead of selecting the ones by presenting a unified architecture which combines real-time object detection based on convolutional neural network, local style transfer using generative adversarial network and steganography together to realize real-time carrier image generation. The object in the carrier image is first detected using a fast object detector and then the detected area is reconstructed through a local generative network. The secret message is embedded into the intermediate generated images during the training process in order to generate an image which is suitable as an image carrier. The experimental results show that the reconstructed stego images are nearly indistinguishable to both human eyes and steganalysis tools. Furthermore, the whole carrier image generation process with GPU implementation can achieve around 5 times faster than the regular CPU implementation which meets the requirement of real-time image processing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Adie, H.T.R., Pradana, I.A., et al.: Parallel computing accelerated image inpainting using gpu cuda, theano, and tensorflow. In: 2018 10th International Conference on Information Technology and Electrical Engineering (ICITEE), pp. 621–625. IEEE (2018)

  2. Baluja, S.: Hiding images in plain sight: deep steganography. In: Advances in Neural Information Processing Systems, pp. 2069–2079 (2017)

  3. Chu, C., Zhmoginov, A., Sandler, M.: Cyclegan: a master of steganography (2017). arXiv:1712.02950

  4. Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 341–346. ACM (2001)

  5. Gatys, L.A., Ecker, A.S., Bethge, M.: A neural algorithm of artistic style (2015). arXiv:1508.06576

  6. Girshick, R.: Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)

  7. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)

  8. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

  9. Hayes, J., Danezis, G.: Generating steganographic images via adversarial training. In: Advances in Neural Information Processing Systems, pp. 1954–1963 (2017)

  10. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988. IEEE (2017)

  11. Herout, A., Jošth, R., Juránek, R., Havel, J., Hradiš, M., Zemčík, P.: Real-time object detection on cuda. J. Real Time Image Proc. 6(3), 159–170 (2011)

    Article  Google Scholar 

  12. Hu, D., Wang, L., Jiang, W., Zheng, S., Li, B.: A novel image steganography method via deep convolutional generative adversarial networks. IEEE Access 6, 38303–38314 (2018)

    Article  Google Scholar 

  13. Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Trans. Gr. 36(4), 107 (2017)

    Article  Google Scholar 

  14. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. arXiv preprint (2017)

  15. Jarušek, R., Volna, E., Kotyrba, M.: Neural network approach to image steganography techniques. In: Mendel 2015, pp. 317–327. Springer (2015)

  16. Li, C., Jiang, Y., Cheslyar, M.: Embedding image through generated intermediate medium using deep convolutional generative adversarial network. CMC Comput. Mater. Continua 56(2), 313–324 (2018)

    Google Scholar 

  17. Li, F., Wu, K., Zhang, X., Lei, J., Wen, M.: Multi-source stego detection with low-dimensional textural feature and clustering ensembles. Symmetry 10(5), 128 (2018)

    Article  Google Scholar 

  18. Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., Yang, M.H.: Universal style transfer via feature transforms. In: Advances in Neural Information Processing Systems, pp. 386–396 (2017)

  19. Li, Y., Liu, M.Y., Li, X., Yang, M.H., Kautz, J.: A closed-form solution to photorealistic image stylization (2018). arXiv:1802.06474

  20. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: European Conference on Computer Vision, pp. 740–755. Springer (2014)

  21. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: Ssd: Single shot multibox detector. In: European Conference on Computer Vision, pp. 21–37. Springer (2016)

  22. Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., Frey, B.: Adversarial autoencoders (2015). arXiv:1511.05644

  23. Meng, R., Rice, S.G., Wang, J., Sun, X.: A fusion steganographic algorithm based on faster R-CNN. Comput. Mater. Continua 55(1), 1–16 (2018)

    Google Scholar 

  24. Papadopoulos, N.A., Psannis, K.E.: Sequential multiple lsb methods and real-time data hiding: variations for visual cryptography ciphers. J. Real Time Image Proc. 14(1), 75–86 (2018)

    Article  Google Scholar 

  25. Qian, Y., Dong, J., Wang, W., Tan, T.: Deep learning for steganalysis via convolutional neural networks. In: Media Watermarking, Security, and Forensics 2015, vol. 9409, p. 94090J. International Society for Optics and Photonics (2015)

  26. Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks (2015). arXiv:1511.06434

  27. Rahim, R., Nadeem, M.S., et al.: End-to-end trained cnn encode-decoder networks for image steganography (2017). arXiv:1711.07201

  28. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

  29. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: Towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)

  30. Sauer, C., Kaplan, R., Lin, A.: Neural fill: content aware image fill with generative adversarial neural networks (2016)

  31. Volkhonskiy, D., Nazarov, I., Borisenko, B., Burnaev, E.: Steganographic generative adversarial networks (2017). arXiv:1703.05502

  32. Wang, X., Zhang, W., Wu, X., Xiao, L., Qian, Y., Fang, Z.: Real-time vehicle type classification with deep convolutional neural networks. J. Real Time Image Proc. 16(1), 5–14 (2019)

    Article  Google Scholar 

  33. Wu, H., Liu, Q., Liu, X.: A review on deep learning approaches to image classification and object segmentation. Comput. Mater. Continua 60(2), 575–597 (2019)

    Article  Google Scholar 

  34. Yang, Y., Wu, J.Q., Feng, X., Thangarajah, A.: Recomputation of dense layers for the performance improvement of DCNN. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (2019). https://doi.org/10.1109/TPAMI.2019.2917685

  35. Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions (2015). arXiv:1511.07122

  36. Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., Huang, T.S.: Free-form image inpainting with gated convolution (2018). arXiv:1806.03589

  37. Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., Huang, T.S.: Generative image inpainting with contextual attention (2018).

  38. Yuan, C., Xia, Z., Jiang, L., Cao, Y., Wu, Q.J., Sun, X.: Fingerprint liveness detection using an improved cnn with image scale equalization. IEEE Access 7, 26953–26966 (2019)

    Article  Google Scholar 

  39. Yuan, C., Xia, Z., Sun, X., Wu, Q.J.: Deep residual network with adaptive learning framework for fingerprint liveness detection. In: IEEE Transactions on Cognitive and Developmental Systems (2019)

  40. Zhang, R., Dong, S., Liu, J.: Invisible steganography via generative adversarial networks. Multimed. Tools Appl. 3, 1–17 (2018)

    Google Scholar 

  41. Zhi, X., Yan, J., Hang, Y., Wang, S.: Realization of cuda-based real-time registration and target localization for high-resolution video images. J. Real Time Image Process. 3, 1–12 (2016)

    Google Scholar 

  42. Zhou, H., Chen, K., Zhang, W., Qian, Z., Yu, N.: Targeted attack and security enhancement on texture synthesis based steganography. J. Vis. Commun. Image Represent. 54, 100–107 (2018)

    Article  Google Scholar 

  43. Zhou, Z., Cao, Y., Wang, M., Fan, E., Wu, Q.J.: Faster-rcnn based robust coverless information hiding system in cloud environment. IEEE Access 7, 179891–179897 (2019)

    Article  Google Scholar 

  44. Zhou, Z., Mu, Y., Wu, Q.J.: Coverless image steganography using partial-duplicate image retrieval. Soft. Comput. 23(13), 4927–4938 (2019)

    Article  Google Scholar 

  45. Zhou, Z., Wu, J.Q., Sun, X.: Multiple distance-based coding: toward scalable feature matching for large-scale web image search. In: IEEE Transactions on Big Data (2019)

Download references

Acknowledgements

This work was supported by the National Key R&D Program of China under grant 2018YFB1003205; by the National Natural Science Foundation of China under Grant U1836208, U1536206, U1836110, 61972205, 61602253, 61672294; by the Jiangsu Basic Research Programs-Natural Science Foundation under Grant numbers BK20181407; by the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD) fund; by the Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET) fund, China. And we further wish to thank Mr. Junhao Cai for his language checking and discussions about the early versions of this work.

Author information

Authors and Affiliations

  1. School of Computer and Software, Nanjing University of Information Science and Technology, Ning Liu Road, No. 219, Nanjing, 210044, China

    Chuanlong Li, Xingming Sun & Zhili Zhou

  2. Jiangsu Engineering Centre of Network Monitoring, Ning Liu Road, No. 219, Nanjing, 210044, China

    Chuanlong Li, Xingming Sun & Zhili Zhou

  3. Department of Computer Science, Lakehead University, Thunder Bay, Canada

    Yimin Yang

Authors
  1. Chuanlong Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xingming Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Zhili Zhou
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Yimin Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding authors

Correspondence to Xingming Sun or Zhili Zhou.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, C., Sun, X., Zhou, Z. et al. Real-time image carrier generation based on generative adversarial network and fast object detection. J Real-Time Image Proc 17, 655–665 (2020). https://doi.org/10.1007/s11554-020-00969-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-020-00969-w

Keywords