Skip to main content

Advertisement

Springer Nature Link
Log in

Hybrid features and semantic reinforcement network for image forgery detection

  • Special Issue Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Image forgery detection focuses more on tampering regions than image content of semantic segmentation, it is revealed that wealthier features need to be learned. Moreover, insufficient semantic information causes low efficiency of forgery detection. To address these issues, we propose a hybrid features and semantic reinforcement network (HFSRNet) for image forgery detection, which is an encoding and decoding based network. Specifically, long-short term memory with resampling features has been applied to capture traces from the image patches for finding manipulating artifacts. Consolidated features extracted from rotating residual units are further leveraged to amplify the discrepancy between un-tampered and tampered regions. We then hybridize features from them through a concatenation to further incorporate spatial co-occurrence of these two modalities. In addition, for achieving the semantic consistency between two same level features associated by across layers, semantic reinforcement is implemented on the decoding stage. HFSRNet is an end-to-end architecture that handles multiple types of image forgery including copy-move, splicing, removal. Experiments on three standard image manipulation datasets (NIST16, COVERAGE and CASIA) demonstrate that HFSRNet obtains state-of-the-art performance compared to existing models and baselines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Cozzolino, D., Poggi, G., Verdoliva, L.: Efficient dense-field copy-move forgery detection. IEEE Trans. Inf. Forens. Secur. 10(11), 2284–2297 (2015)

    Article  Google Scholar 

  2. Wen, B.H., Zhu, Y., Subramanian, R., Ng, T.T., Shen, X.J., Winkler, S.: Coverage a novel database for copy-move forgery detection. In: IEEE International Conference on Image processing (ICIP), pp. 161–165 (2016)

  3. Cozzolino, D., Poggi, G., Verdoliva, L.: Splicebuster: A new blind image splicing detector. In: IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1–6 (2015)

  4. Wu, Y., Abd-Almageed, W., Natarajan, P.: Deep matching and validation network: an end-to-end solution to constrained image splicing localization and detection. In: ACM on Multimedia Conference (MM), pp. 1480–1502 (2017)

  5. Zhu, X.S., Qian, Y.J., Zhao, X.F., Sun, B., Sun, Y.: A deep learning approach to patch-based image inpainting forensics. Signal Process. Image Commun. 67, 90–99 (2018)

    Article  Google Scholar 

  6. Bayar, B., C, M.: A deep learning approach to universal image manipulation detection using a new convolutional layer. In: ACM Workshop on Information Hiding & Multimedia Security, pp. 5–10 (2016)

  7. Han, J.G., Park, T.H., Moon, Y.H., Eom, I.K.: Efficient Markov feature extraction method for image splicing detection using maximization and threshold expansion. J. Electron. Imaging 25(2), 023031 (2016)

    Article  Google Scholar 

  8. Asghar, K., Habib, Z., Hussain, M.: Copy-move and splicing image forgery detection and localization techniques: a review. Aust. J. Forensic Sci 49(3), 281–307 (2017)

    Article  Google Scholar 

  9. Yang, Q., Peng, F., Li, J.-T., Long, M.: Image tamper detection based on noise estimation and lacunarity texture. Multim. Tools Appl. 75(17), 10201–10211 (2016)

    Article  Google Scholar 

  10. Girshick, R.: Fast R-CNN. In: International Conference on Computer Vision (ICCV), pp. 1440–1448 (2015)

  11. Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., Oliva, A.: Learning deep features for scene recognition using places database. In: Advances in neural information processing systems (NIPS), pp. 487–495 (2014)

  12. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440 (2015)

  13. Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., Torr, P.H.: Conditional random fields as recurrent neural networks. In: International Conference on Computer Vision (ICCV), pp. 1529–1537 (2015)

  14. Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: a deep convolutional encoder-decoder architecture for scene segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)

    Article  Google Scholar 

  15. Rao, Y., Ni, J.Q.: A deep learning approach to detection of splicing and copy-move forgeries in images. In: IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1–6 (2016)

  16. Bi,X., Wei, Y., Xiao, B., Li, W.: RRU-Net: The Ringed Residual U-Net for Image Splicing Forgery Detection. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 30–39 (2019)

  17. Bappy, J.H., Roy-Chowdhury, A.K., Bunk, J., Nataraj, L., Manjunath, B.: Exploiting spatial structure for localizing manipulated image regions. In: International Conference on Computer Vision (ICCV), pp. 4980–4989 (2017)

  18. Bappy, J.H., Simons, C., Nataraj, L., Manjunath, B.S., Roy-Chowdhury, A.K.: Hybrid LSTM and encoder-decoder architecture for detection of image forgeries. IEEE Trans. Image Process. 28(7), 3286–3300 (2019)

    Article  MathSciNet  Google Scholar 

  19. NIST manipulation evaluation dataset. https://www.nist.gov/itl/iad/mig/nimble-challenge-2017-evaluation/ (2016). Accessed 4 Aug 2017

  20. Dong, J., Wang, W., Tan, T.: Casia image tampering detection evaluation database. In: 2013 IEEE China Summit and International Conference on Signal and Information Processing (ChinaSIP), pp. 422–426 (2013)

  21. Mahdian, B., Saic, S.: Using noise inconsistencies for blind image forensics. Image Vis. Comput. 27(10), 1497–1503 (2009)

    Article  Google Scholar 

  22. Nataraj, L., Sarkar, A., Manjunath, B.S.: Improving re-sampling detection by adding noise. In: Conference on Media Forensics and Security II. 7541, 75410I (2010)

  23. Krawetz, N.: A picture’s worth... Hacker Factor Solutions. 6 (2007)

  24. Ferrara, P., Bianchi, T., De Rosa, A., Piva, A.: Image forgery localization via fine-grained analysis of cfa artifacts. IEEE Trans. Inf. Forensics Secur. 7(5), 1566–1577 (2012)

    Article  Google Scholar 

  25. Salloum, R., Ren, Y., et al.: Image splicing localization using a multi-task fully convolutional network (mfcn). J. Vis. Commun. Image Represent. 51, 201–209 (2018)

    Article  Google Scholar 

  26. Tan, H.C., Liu, X.P., Liu, M., Yin, B.C., Li, X.: KT-GAN: knowledge-transfer generative adversarial network for text-to-image synthesis. IEEE Trans. Image Process. 30, 1275–1290 (2021)

    Article  Google Scholar 

  27. Fridrich, J., Kodovsky, J.: Rich models for steganalysis of digital images. IEEE Trans. Inf. Forensics Secur. 7(3), 868–882 (2012)

    Article  Google Scholar 

  28. Bayar, B., Stamm, M.C.: Constrained convolutional neural networks: a new approach towards general purpose image manipulation detection. IEEE Trans. Inf. Forensics Secur. 13(11), 2691–2706 (2018)

    Article  Google Scholar 

  29. Zhou, P., Han, X., Morariu, V.I., Davis, L.S.: Learning rich features for image manipulation detection. In: Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1053–1061 (2018)

  30. Huizhou, L., Chao, Y., Fangting, L., Bin, J.: Constrained R-CNN: a general image manipulation detection model. In: International Conference on Multimedia and Expo (ICME), pp. 1–6 (2020)

  31. Li, J., Li, X., Yang, B., Sun, X.: Segmentation-based image copy-move forgery detection scheme. IEEE Trans. Inf. Forensics Secur. 10(3), 507–518 (2015)

    Article  Google Scholar 

  32. Bunk, J., Bappy, J.H., Mohammed, T.M., Nataraj, L., Flenner, A., Manjunath, B.S., Chandrasekaran, S., Roy-Chowdhury, A.K., Peterson, L.: Detection and Localization of Image Forgeries Using Resampling Features and Deep Learning. In: Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1881–1889 (2017)

  33. Moon, B., Jagadish, H.V., Faloutsos, C., Saltz, J.H.: Analysis of the clustering properties of the Hilbert space-filling curve. IEEE Trans. Knowl. Data Eng. 13(1), 124–141 (2001)

    Article  Google Scholar 

  34. Kingma, D., Ba, J.: Adam: A method for stochastic optimization. In: International Conference on Learning Representations (2014)

  35. Gloe, T., Bohme, R.: The Dresden image database for benchmarking digital image forensics. J. Dig. Forensic Pract. 3(2–4), 150–159 (2010)

    Article  Google Scholar 

  36. Mazaheri, G., Mithun, N.C., Bappy, J.H., Roy-Chowdhury, A.K.: A skip connection architecture for localization of image manipulations. In: Proceedings, IEEE Conference Computing Vision, Pattern Recognition, pp. 119–129 (2019)

Download references

Acknowledgements

This research is supported by the National Key Research and Development Program of China (2018YFB0804203), Regional Joint Fund of NSFC (U19A2057), the National Natural Science Foundation of China (61672259, 61876070), and the Jilin Province Science and Technology Development Plan Project(20190303134SF, 20180201064SF).

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Jilin University, Changchun, 130012, China

    Haipeng Chen & Zenan Shi

  2. College of Software, Jilin University, Changchun, 130012, China

    Chaoqun Chang

  3. Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012, China

    Haipeng Chen, Chaoqun Chang & Zenan Shi

  4. Public Computer Education and Research Center, Jilin University, Changchun, 130012, China

    Yingda Lyu

Authors
  1. Haipeng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chaoqun Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zenan Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yingda Lyu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yingda Lyu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, H., Chang, C., Shi, Z. et al. Hybrid features and semantic reinforcement network for image forgery detection. Multimedia Systems 28, 363–374 (2022). https://doi.org/10.1007/s00530-021-00801-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-021-00801-w

Keywords