Skip to main content
SpringerLink
Log in

A hybrid convolutional architecture for accurate image manipulation localization at the pixel-level

  • 1163: Large-scale multimedia signal processing for security and digital forensics
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

A Correction to this article was published on 23 July 2022

This article has been updated

Abstract

Advanced image processing techniques can easily edit images without leaving any visible traces, making manipulation detection and localization for forensics analysis a challenging task. Few studies can simultaneously locate tampered objects accurately and refine contours of tampered regions effectively. In this study, we propose an effective and novel hybrid architecture, named Pixel-level Image Tampering Localization Architecture (PITLArc), which integrates the advantages of top-down detection-based methods and bottom-up segmentation-based methods. Moreover, we provide a typical fusion implementation of our proposed hybrid architecture on one outstanding detection-based method (two-stream faster region-based convolutional neural network (RGB-N)) and two segmentation-based methods (Multi-Scale Convolution Neural Networks (MSCNNs) and Dual-domain Convolutional Neural Networks (DCNNs)) to evaluate the effectiveness of the proposed architecture. The three methods can be integrated into our proposed PITLArc to significantly improve their performance. Other detection and segmentation algorithms (not limited to the three aforementioned methods) can also be integrated into our architecture to improve their performance. Moreover, a Dense Conditional Random Fields (DenseCRFs)-based post-processing method is introduced to further optimize the details of tampered regions. Experiments validate the effectiveness of the proposed architecture.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Change history

References

  1. Badrinarayanan V, Kendall A, Cipolla R (2015) Segnet: A deep convolutional encoder-decoder architecture for image segmentation. CoRR 1511.00561

  2. Bahrami K, Kot AC (2017) Image splicing localization based on blur type inconsistency. IEEE Trans Inf Forensic Secur 10(5):999–1009

    Article  Google Scholar 

  3. Bappy MJH, Roy-Chowdhury AK, Bunk J, Nataraj L, Manjunath BS (2017) Exploiting spatial structure for localizing manipulated image regions. In: IEEE International Conference on Computer Vision

  4. Bappy JH, Simons C, Nataraj L, Manjunath BS, Roy-Chowdhury AK (2019) Hybrid lstm and encoder–decoder architecture for detection of image forgeries. IEEE Trans Image Process 28(7):3286–3300

    Article  MathSciNet  Google Scholar 

  5. Bayar B, Stamm MC (2016) A deep learning approach to universal image manipulation detection using a new convolutional layer. In: ACM Workshop on Information Hiding and Multimedia Security

  6. Bianchi T, Rosa AD, Piva A (2011) Improved dct coefficient analysis for forgery localization in jpeg images. In: IEEE International Conference on Acoustics

  7. Bianchi T, Piva A (2012) Image forgery localization via block-grained analysis of jpeg artifacts. IEEE Trans Inf Forensic Secur 7(3):1003–1017

    Article  Google Scholar 

  8. Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille A L Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs

  9. Chen M, Fridrich JJ, Goljan M, Lukás J (2008) Determining image origin and integrity using sensor noise. IEEE Trans Inf Forensic Secur 3(1):74–90

    Article  Google Scholar 

  10. Chen J, Kang X, Ye L, Wang ZJ (2015) Median filtering forensics based on convolutional neural networks. IEEE Signal Process Lett 22(11):1849–1853

    Article  Google Scholar 

  11. Chierchia G, Poggi G, Sansone C, Verdoliva L (2014) A bayesian-mrf approach for prnu-based image forgery detection. IEEE Trans Inf Forensic Secur 9(4):554–567

    Article  Google Scholar 

  12. Cozzolino D, Poggi G, Verdoliva L (2016) Splicebuster: a new blind image splicing detector. In: IEEE International Workshop on Information Forensics and Security

  13. Dirik AE, Memon N (2009) Image tamper detection based on demosaicing artifact. In: IEEE International Conference on Image Processing

  14. Ferrara P, Bianchi T, Rosa AD, Piva A (2012) Image forgery localization via fine-grained analysis of cfa artifacts. IEEE Trans Inf Forensic Secur 7(5):1566–1577

    Article  Google Scholar 

  15. Fridrich J, Kodovsky J (2012) Rich models for steganalysis of digital images. IEEE Trans Inf Forensic Secur 7(3):868–882

    Article  Google Scholar 

  16. Gao Y, Beijbom O, Zhang N, Darrell T (2016) Compact bilinear pooling. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 317–326

  17. Kakar P, Natarajan S, Ser W (2010) Detecting digital image forgeries through inconsistent motion blur. Icme, pp 486–491

  18. Krähenbähl P, Koltun V Efficient inference in fully connected crfs with gaussian edge potentials

  19. Li H, Luo W, Qiu X, Huang J (2017) Image forgery localization via integrating tampering possibility maps. IEEE Trans Inf Forensic Secur 12(5):1240–1252

    Article  Google Scholar 

  20. Lin T-Y, RoyChowdhury A, Maji S (2015) Bilinear cnn models for fine-grained visual recognition. In: Proceedings of the IEEE international conference on computer vision, pp 1449–1457

  21. Lin Z, He J, Tang X, Tang CK (2009) Fast, automatic and fine-grained tampered jpeg image detection via dct coefficient analysis?. Pattern Recogn 42(11):2492–2501

    Article  Google Scholar 

  22. Liu Y, Guan Q, Zhao X, Yun C (2018) Image forgery localization based on multi-scale convolutional neural networks

  23. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  24. Lyu S, Pan X, Xing Z (2014) Exposing region splicing forgeries with blind local noise estimation. Int J Comput Vis 110(2):202–221

    Article  Google Scholar 

  25. Mahdian B, Saic S (2009) Using noise inconsistencies for blind image forensics. Image Vis Comput 27(10):1497–1503

    Article  Google Scholar 

  26. PyDenseCRFs. https://github.com/lucasb-eyer/pydensecrf

  27. Ren S, He K, Girshick RB, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. CoRR 1506.01497

  28. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-assisted Intervention

  29. Salloum R, Ren Y, Kuo CCJ (2017) Image splicing localization using a multi-task fully convolutional network (mfcn). J Vis Commun Image Represent 51:201–209

    Article  Google Scholar 

  30. Shi Z, Shen X, Kang H, Lv Y (2018) Image manipulation detection and localization based on the dual-domain convolutional neural networks. IEEE Access

  31. Ying Zhang LLWVT (2016) Image region forgery detection: A deep learning approach, vol 14. https://doi.org/10.3233/978-1-61499-617-0-1

  32. Yuan R, Ni J (2017) A deep learning approach to detection of splicing and copy-move forgeries in images. IEEE International Workshop on Information Forensics and Security

  33. Zhang Y, Thing VLL (2018) A semi-feature learning approach for tampered region localization across multi-format images. Multimed Tools Appl 77(19):25027–25052

    Article  Google Scholar 

  34. Zhou P, Han X, Morariu VI, Davis LS (2018) Learning rich features for image manipulation detection. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Download references

Acknowledgments

This work was supported by NSFC under U1636102, U1736214, 61802393 and 61872356, National Key Technology R&D Program under 2016QY15Z2500, and Project of Beijing Municipal Science & Technology Commission under Z181100002718001.

Author information

Authors and Affiliations

  1. School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China

    Yixuan Zhang

  2. State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China

    Yixuan Zhang

  3. National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Jiguang Zhang & Shibiao Xu

Authors
  1. Yixuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiguang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shibiao Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shibiao Xu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original online version of this article was revised: In the second paragraph of section 4.2, the words "pristine" and "manipulated" were interchangeably used.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Zhang, J. & Xu, S. A hybrid convolutional architecture for accurate image manipulation localization at the pixel-level. Multimed Tools Appl 80, 23377–23392 (2021). https://doi.org/10.1007/s11042-020-10211-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10211-1

Keywords

Search

Navigation