Skip to main content
Springer Nature Link
Log in

MSU-Net: the multi-scale supervised U-Net for image splicing forgery localization

  • Original Article
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Image splicing forgery, that is, copying some parts of an image into another image, is one of the frequently used tampering methods in image forgery. As a research hotspot in recent years, deep learning has been used in image forgery detection. However, current deep learning methods have two drawbacks: first, they are too simple in feature fusion; second, they rely only on a single cross-entropy loss as the loss function, leading to models prone to overfitting. To address these issues, a image splicing forgery localization method based on multi-scale supervised U-shaped network, named MSU-Net, is proposed in this paper. First, a triple-stream feature extraction module is designed, which combines the noise view and edge information of the input image to extract semantic-related and semantic-agnostic features. Second, a feature hierarchical fusion mechanism is proposed that introduces a channel attention mechanism layer by layer to perceive multi-level manipulation trajectories, avoiding the loss of information in semantic-related and semantic-agnostic shallow features during the convolution process. Finally, a strategy for multi-scale supervision is developed, a boundary artifact localization module is designed to compute the edge loss, and a contrastive learning module is introduced to compute the contrastive loss. Through extensive experiments on several public datasets, MSU-Net demonstrates high accuracy in localizing tampered regions and outperforms state-of-the-art methods. Additional attack experiments show that MSU-Net exhibits good robustness against Gaussian blur, Gaussian noise, and JPEG compression attacks. Besides, MSU-Net is superior in terms of model complexity and localization speed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from the corresponding author, upon reasonable request.

References

  1. Cozzolino D, Poggi G, Verdoliva L (2015) Splicebuster: a new blind image splicing detector. In: 2015 IEEE international workshop on information forensics and security (WIFS). IEEE, pp 1–6

  2. Kniaz VV, Knyaz V, Remondino F (2019) The point where reality meets fantasy: mixed adversarial generators for image splice detection. In: Advances in neural information processing systems, vol 32

  3. Wu Y, Abd-Almageed W, Natarajan P (2018) Image copy-move forgery detection via an end-to-end deep neural network. In: 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 1907–1915

  4. Zhu X, Qian Y, Zhao X, Sun B, Sun Y (2018) A deep learning approach to patch-based image inpainting forensics. Signal Process Image Commun 67:90–99

    Article  Google Scholar 

  5. Chen X, Dong C, Ji J, Cao J, Li X (2021) Image manipulation detection by multi-view multi-scale supervision. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 14185–14193

  6. Wei Y, Ma J, Wang Z, Xiao B, Zheng W (2022) Image splicing forgery detection by combining synthetic adversarial networks and hybrid dense u-net based on multiple spaces. Adv Neural Inf Process Syst 37(11):8291–8308

    Google Scholar 

  7. Fridrich J, Kodovsky J (2012) Rich models for steganalysis of digital images. IEEE Trans Inf Forensics Secur 7(3):868–882

    Article  Google Scholar 

  8. Bi X, Wei Y, Xiao B, Li W (2019) RRU-Net: the ringed residual u-net for image splicing forgery detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW), pp 30–39

  9. Yang C, Li H, Lin F, Jiang B, Zhao H (2020) Constrained R-CNN: a general image manipulation detection model. In: 2020 IEEE international conference on multimedia and expo (ICME). IEEE, pp 1–6

  10. Xu Y, Zheng J, Fang A, Irfan M (2023) Feature enhancement and supervised contrastive learning for image splicing forgery detection. Digit Signal Process 136:1–17

    Article  Google Scholar 

  11. Zhuo L, Tan S, Li B, Huang J (2022) Self-adversarial training incorporating forgery attention for image forgery localization. IEEE Trans Inf Forensics Secur 17:819–834

    Article  Google Scholar 

  12. Wu H, Zhou J, Tian J, Liu J, Qiao Y (2022) Robust image forgery detection against transmission over online social networks. IEEE Trans Inf Forensics Secur 17:443–456

    Article  Google Scholar 

  13. Zhou P, Han X, Morariu VI, Davis LS (2018) Learning rich features for image manipulation detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1053–1061

  14. Wu Y, AbdAlmageed W, Natarajan P (2019) Mantra-net: Manipulation tracing network for detection and localization of image forgeries with anomalous features. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 9543–9552

  15. Hu X, Zhang Z, Jiang Z, Chaudhuri S, Yang Z, Nevatia R (2020) Span: Spatial pyramid attention network for image manipulation localization. In: European conference on computer vision. Springer, Berlin, pp 312–328

  16. Kwon M, Yu I, Nam S, Lee H (2021) CAT-Net: Compression artifact tracing network for detection and localization of image splicing. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 375–384

  17. Niloy FF, Bhaumik KK, Woo SS (2023) CFL-Net: Image forgery localization using contrastive learning. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 4642–4651

  18. Zhang Y, Zhao C, Pi Y, Li S (2012) Revealing image splicing forgery using local binary patterns of DCT coefficients. In: Communications signal processing and systems. Springer, New York, pp 181–189

  19. Alahmadi A, Hussain M, Aboalsamh H, Muhammad G, Bebis G, Mathkour H (2017) Passive detection of image forgery using DCT and local binary pattern. Signal Image Video Process 11(1):81–88

    Article  Google Scholar 

  20. Zhang Q, Wei L, Weng J (2016) Joint image splicing detection in DCT and contourlet transform domain. J Vis Commun Image Represent 40:449–458

    Article  Google Scholar 

  21. Lin Z, He J, Tang X, Tang CK (2009) Fast, automatic and fine-grained tampered jpeg image detection via DCT coefficient analysis. Pattern Recognit 42(11):2492–2501

    Article  Google Scholar 

  22. Mire AV, Dhok SB, Mistry NJ, Porey PD (2018) Automated approach for splicing detection using first digit probability distribution features. EURASIP J Image Video Process 2018(1):1–11

    Article  Google Scholar 

  23. Amerini I, Becarelli R, Caldelli R, Mastio AD (2014) Splicing forgeries localization through the use of first digit features. In: 2014 IEEE international workshop on information forensics and security (WIFS). IEEE, pp 143–148

  24. Iakovidou C, Zampoglou M, Papadopoulos S, Kompatsiaris Y (2018) Content-aware detection of jpeg grid inconsistencies for intuitive image forensics. J Vis Commun Image Represent 54:155–170

    Article  Google Scholar 

  25. Zeng H, Zhan Y, Kang X, Lin X (2017) Image splicing localization using pca-based noise level estimation. Multimed Tools Appl 76(4):4783–4799

    Article  Google Scholar 

  26. Yao H, Wang S, Zhang X, Qin C, Wang J (2017) Detecting image splicing based on noise level inconsistency. Multimed Tools Appl 76(10):12457–12479

    Article  Google Scholar 

  27. Zhu N, Li Z (2018) Blind image splicing detection via noise level function. Signal Process Image Commun 68:181–192

    Article  Google Scholar 

  28. Tagliasacchi M, Valenzise G, Tubaro S (2009) Hash-based identification of sparse image tampering. IEEE Trans Image Process 18(11):2491–2504

    Article  MathSciNet  Google Scholar 

  29. Zhao Y, Wang S, Zhang X, Yao H (2012) Robust hashing for image authentication using zernike moments and local features. IEEE Trans Inf Forensics Secur 8(1):55–63

    Article  Google Scholar 

  30. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 234–241

  31. Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H (2019) Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 3146–3154

  32. Khosla P, Teterwak P, Wang C, Sarna A, Tian Y, Isola P, Maschinot A, Liu C, Krishnan D (2020) Supervised contrastive learning. Adv Neural Inf Process Syst 33:18661–18673

    Google Scholar 

  33. Dong J, Wang W, Tan T (2013) Casia image tampering detection evaluation database. In: 2013 IEEE China summit and international conference on signal and information processing. IEEE, pp 422–426

  34. Dong J, Wang W, Tan T (2010) Casia image tampering detection evaluation database. http://forensics.idealtest.org

  35. Ng T-T, Hsu J, Chang S-F (2009) Columbia image splicing detection evaluation dataset. Columbia Univ CalPhotos Digit Libr, DVMM lab

  36. Guan H, Kozak M, Robertson E, Lee Y, Yates AN, Delgado A, Zhou D, Kheyrkhah T, Smith J, Fiscus J (2019) MFC datasets: large-scale benchmark datasets for media forensic challenge evaluation. In: 2019 IEEE winter applications of computer vision workshops (WACVW). IEEE, pp 63–72

  37. Huh M, Liu A, Owens A, Efros AA (2018) Fighting fake news: image splice detection via learned self-consistency. In: Proceedings of the European conference on computer vision (ECCV), pp 101–117

  38. Kingma D, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  39. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167

  40. Wu Y, He K (2018) Group normalization. arXiv preprint arXiv:1803.08494

  41. Krawetz N, Solution H (2007) A picture’s worth. Hacker factor. Solutions 6(2):2

    Google Scholar 

  42. Mahdian B, Stanislav S (2009) Using noise inconsistencies for blind image forensics. Image Vis Comput 27(10):1497–1503

    Article  Google Scholar 

  43. Ferrara P, Bianchi T, Rosa AD, Piva A (2012) Image forgery localization via fine-grained analysis of CFA artifacts. IEEE Trans Inf Forensics Secur 7(5):1566–1577

    Article  Google Scholar 

  44. Zampoglou M, Papadopoulos S, Kompatsiaris Y (2017) Large-scale evaluation of splicing localization algorithms for web images. Multimed Tools Appl 76(4):4801–4834

    Article  Google Scholar 

Download references

Funding

The paper is supported by the Natural Science Foundation of Fujian province, China (2022J05028)

Author information

Authors and Affiliations

  1. College of Computer and Data Science, Fuzhou University, Fuzhou, 350116, China

    Hao Yu, Lichao Su, Chenwei Dai & Jinli Wang

Authors
  1. Hao Yu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Lichao Su
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Chenwei Dai
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Jinli Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Hao Yu: Methodology, Software, Conceptualization, Writing - original draft. Lichao Su: Project administration, Supervision, Writing - review & editing. Chenwei Dai: Formal analysis, Data curation, Validation. Jinli Wang: Investigation, Visualization.

Corresponding author

Correspondence to Lichao Su.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest that are relevant to the content of this article.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, H., Su, L., Dai, C. et al. MSU-Net: the multi-scale supervised U-Net for image splicing forgery localization. Pattern Anal Applic 27, 86 (2024). https://doi.org/10.1007/s10044-024-01305-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10044-024-01305-9

Keywords