Skip to main content
SpringerLink
Log in

RB-Net: integrating region and boundary features for image manipulation localization

  • Special Issue Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Current research on image tampering localization focuses on finding region features that distinguish manipulated pixels from non-manipulated pixels. As tampering with a specific area of a given image inevitably leaves cues in the boundary between the tampered region and its surroundings, how to utilize sufficient region and boundary features also matters for image manipulation localization. In this paper, we propose a unified network (called RB-Net), which is a two-branch network (i.e., region module and boundary module) to learn region and boundary features separately. Then the fusion module is implemented to integrate the region features from the region module and the edge features from the boundary module, respectively. Particularly, to identify unnatural boundary traces, we propose edge gate components deployed on different layers of the region module to activate manipulated boundary information from the rich region features. Quantitative and qualitative experiments on four benchmark datasets demonstrate that RB-Net can accurately locate the tampered regions and achieve the best results relative to other state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Zhang, J., Wang, M., Lin, L., Yang, X., Gao, J., Rui, Y.: Saliency detection on light field: a multi-cue approach. Trans. Multimed. Comput. Commun. Appl. 13(3), 1–22 (2017)

    Article  Google Scholar 

  2. Du, X., Yang, X., Qin, Z., Tang, J.: Progressive image enhancement under aesthetic guidance. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 349–353 (2019)

  3. Liu, X., Yang, X., Wang, M., Hong, R.: Deep neighborhood component analysis for visual similarity modeling. ACM Trans. Intell. Syst. Technol. (TIST) 11(3), 1–15 (2020)

    Google Scholar 

  4. Meng, L., Chen, L., Yang, X., Tao, D., Zhang, H., Miao, C., Chua, T-S.: Learning using privileged information for food recognition. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 557–565 (2019)

  5. Yang, X., Zhou, P., Wang, M.: Person reidentification via structural deep metric learning. IEEE Trans. Neural Netw. Learn. Syst. 30(10), 2987–2998 (2019)

    Article  Google Scholar 

  6. Yang, X., Feng, F., Ji, W., Wang, M., Chua, T-S.: Deconfounded video moment retrieval with causal intervention. In: The 44th International Conference on Research and Development in Information Retrieval (SIGIR), pp. 1–10 (2021)

  7. Ryu, S.-J., Lee, H.-K.: Estimation of linear transformation by analyzing the periodicity of interpolation. Pattern Recognit. Lett. 36(1), 89–99 (2014)

    Article  Google Scholar 

  8. Kwon, Y., Kim, K.I., Tompkin, J., Kim, J.H., Theobalt, C.: Efficient learning of image superresolution and compression artifact removal with semi-local Gaussian processes. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1792–1805 (2015)

    Article  Google Scholar 

  9. Li, J., Li, X., Yang, B., Sun, X.: Segmentation-based image copy-move forgery detection scheme. IEEE Trans. Inf. Forensics Secur. 10(3), 507–518 (2015)

    Article  Google Scholar 

  10. Wu, Y., Abd-Almageed, W., Natarajan, P.: Busternet: Detecting copy-move image forgery with source/target localization. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 168–184 (2018)

  11. Zhu, Y., Chen, C., Yan, G., Guo, Y., Dong, Y.: AR-Net: adaptive attention and residual refinement network for copy-move forgery detection. IEEE Trans. Ind. Inf. 16(10), 6714–6723 (2020). https://doi.org/10.1109/TII.2020.2982705

    Article  Google Scholar 

  12. Manu, V., Mehtre, B.: Visual artifacts based image splicing detection in uncompressed images. In: IEEE International Conference on Computer Graphics, Vision and Information Security (CGVIS) (2015)

  13. Cun, X., Pun, C.-M.: Image splicing localization via semi-global network and fully connected conditional random fields. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 252–266 (2018)

  14. Liang, Z., Yang, G., Ding, X., Li, L.: An efficient forgery detection algorithm for object removal by exemplar-based image inpainting. J. Vis. Commun. Image Represent. 30, 75–85 (2015)

    Article  Google Scholar 

  15. Li, H., Luo, W., Huang, J.: Localization of diffusion-based inpainting in digital images. IEEE Trans. Inf. Forensics Secur. 12(12), 3050–3064 (2017). https://doi.org/10.1109/TIFS.2017.2730822

    Article  Google Scholar 

  16. Shi, Z., Shen, X., Kang, H., Lyu, Y.: Image manipulation detection and localization based on the dual-domain convolutional neural networks. IEEE J. Transl. Eng. Health Med. 6, 76437–76453 (2018)

    Google Scholar 

  17. Shi, Z., Shen, X., Chen, H., Lyu, Y.: Global semantic consistency network for image manipulation detection. IEEE Signal Process Lett. 27, 1755–1759 (2020). https://doi.org/10.1109/LSP.2020.3026954

    Article  Google Scholar 

  18. Chen, H., Chang, C., Shi, Z., Lyu, Y.: Hybrid features and semantic reinforcement network for image forgery detection. Multimed. Syst. 11, 1–12 (2021)

    Google Scholar 

  19. Zhou, P., Han, X., Morariu, V.I., Davis, L.S.: Learning rich features for image manipulation detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1053–1061 (2018). https://doi.org/10.1109/CVPR.2018.00116

  20. Wu, Y., AbdAlmageed, W., Natarajan, P.: Mantra-net: manipulation tracing network for detection and localization of image forgeries with anomalous features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9543–9552 (2019).https://doi.org/10.1109/CVPR.2019.00977

  21. Hu, X., Zhang, Z., Jiang, Z., Chaudhuri, S., Yang, Z., Nevatia, R.: SPAN: Spatial pyramid attention network for image manipulation localization. In: Proceedings of the European Conference on Computer Vision (ECCV) (2020)

  22. Yang, C., Li, H., Lin, F., Jiang, B., Zhao, H.: Constrained R-CNN: a general image manipulation detection model. In: 2020 IEEE International Conference on Multimedia and Expo (ICME) (2020). https://doi.org/10.1109/ICME46284.2020.9102825

  23. Mazaheri, G., Mithun, N.C., Bappy, J.H., Roy-Chowdhury, A.K.: A skip connection architecture for localization of image manipulations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 119-129 (2019)

  24. Bappy, J.H., Simons, C., Nataraj, L., Manjunath, B.S., Roy-Chowdhury, A.K.: Hybrid LSTM and encoder–decoder architecture for detection of image forgeries. IEEE Trans. Image Process. 28(7), 3286–3300 (2019). https://doi.org/10.1109/TIP.2019.2895466

    Article  MathSciNet  MATH  Google Scholar 

  25. Zhou, P., Chen, B., Han, X., Najibi, M., Davis, L.: Generate, segment, and refine: towards generic manipulation segmentation. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), vol. 34 (7), pp. 13058–13065 (2020)

  26. Salloum, R., Ren, Y., Kuo, C.-C.J.: Image splicing localization using a multi-task fully convolutional network (MFCN). J. Vis. Commun. Image Represent. 51, 201–209 (2017)

    Article  Google Scholar 

  27. Fridrich, J., Kodovsky, J.: Rich models for steganalysis of digital images. IEEE Trans. Inf. Forensics Secur. 7(3), 868–882 (2012). https://doi.org/10.1109/TIFS.2012.2190402

    Article  Google Scholar 

  28. Bayar, B., Stamm, M.C.: Constrained convolutional neural networks: a new approach towards general purpose image manipulation detection. IEEE Trans. Inf. Forensics Secur. 13(11), 2691–2706 (2018). https://doi.org/10.1109/TIFS.2018.2825953

    Article  Google Scholar 

  29. Liu, B., Pun, C.-M.: Deep fusion network for splicing forgery localization. In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)

  30. Bi, X., Wei, Y., Xiao, B., Li, W.: RRU-Net: the ringed residual U-Net for image splicing forgery detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE (2019). https://doi.org/10.1109/CVPRW.2019.00010

  31. Zhong, J., Pun, C.: An end-to-end dense-InceptionNet for image copy-move forgery detection. IEEE Trans. Inf. Forensics Secur. 15, 2134–2146 (2020)

    Article  Google Scholar 

  32. Barni, M., Phan, Q.T., Tondi, B.: Copy move source-target disambiguation through multi-branch CNNs. IEEE Trans. Inf. Forensics Secur. 16, 1825–1840 (2021). https://doi.org/10.1109/TIFS.2020.3045903

    Article  Google Scholar 

  33. Liu, Y., Zhu, X., Zhao, X., Cao, Y.: Adversarial learning for constrained image splicing detection and localization based on atrous convolution. IEEE Trans. Inf. Forensics Secur. 14(10), 2551–2566 (2019)

    Article  Google Scholar 

  34. Li, H., Huang, J.: Localization of deep inpainting using high-pass fully convolutional network. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 8300–8309 (2019). https://doi.org/10.1109/ICCV.2019.00839

  35. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7132–7141 (2018)

  36. Du, X.-Y., Yang, Y., Yang, L., Shen, F.-M., Qin, Z.-G., Tang, J.-H.: Captioning videos using large-scale image corpus. J. Comput. Sci. Technol. 32(3), 480–493 (2017)

    Article  Google Scholar 

  37. Zhang, D., Zhang, H., Tang, J., Wang, M., Hua, X., Sun, Q.: Feature pyramid transformer. In: European Conference on Computer Vision (ECCV), pp. 323–339 (2020)

  38. Tan, Y., Hao, Y., He, X., Wei, Y., Yang, X.: Selective dependency aggregation for action classification. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 592–601 (2021)

  39. Yang, X., Liu, X., Jian, M., Gao, X., Wang, M.: Weakly-supervised video object grounding by exploring spatio-temporal contexts. In: The 28th ACM International Conference on Multimedia (ACM), pp. 1939–1947 (2020)

  40. Woo, S., Park, J., Lee, J.Y., Kweon, I.S.: CBAM: convolutional block attention module. Proc. Eur. Conf. Comput. Vis. (ECCV) 7, 3–19 (2018)

    Google Scholar 

  41. Takikawa, T., Acuna, D., Jampani, V., Fidler, S.: Gated-scnn: Gated shape cnns for semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision (CVPR), pp. 5229–5238 (2019).https://doi.org/10.1109/ICCV.2019.00533

  42. Zagoruyko, S., Komodakis, N.: Wide residual networks. In: 27th British Machine Vision Conference (BMVC) (2016)

  43. Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder–decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 801–818 (2018)

  44. Zimmermann, R., Siems, J.: Faster training of Mask R-CNN by focusing on instance boundaries. Comput. Vis. Image Underst. 188, 102795 (2019)

    Article  Google Scholar 

  45. NIST: Nimble media forensics challenge datasets (2016). https://www.nist.gov/itl/iad/mig/media-forensics-challenge

  46. Wen, B., Zhu, Y., Subramanian, R., Ng, T.-T., Winkler, S.: COVERAGE-A novel database for copy-move forgery detection. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 161–165 (2016). https://doi.org/10.1109/ICIP.2016.7532339

  47. Dong, J., Wang, W., Tan, T.: Casia image tampering detection evaluation database. In: 2013 IEEE China Summit and International Conference on Signal and Information Processing (ChinaSIP), pp. 422–426 (2013).https://doi.org/10.1109/ChinaSIP.2013.6625374

  48. de Carvalho, T.J., Riess, C., Angelopoulou, E., Pedrini, H., Rocha, A.D.: Exposing digital image forgeries by illumination color classification. IEEE Trans. Inf. Forensics Secur. 8(7), 1182–1194 (2013)

    Article  Google Scholar 

  49. Paszke, A., Gross, S., Chintala, S., et al.: Automatic differentiation in pytorch (2017)

  50. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980, 273–297 (2014)

  51. Mahdian, B., Saic, S.: Using noise inconsistencies for blind image forensics. Image Vis. Comput. 27(10), 1497–1503 (2009)

    Article  Google Scholar 

  52. Ferrara, P., Bianchi, T., Rosa, A.D., Piva, A.: Image forgery localization via fine-grained analysis of CFA artifacts. IEEE Trans. Inf. Forensics Secur. 7(5), 1566–1577 (2012)

    Article  Google Scholar 

  53. Krawetz, N., Solutions, H.F.: A picture’s worth. Hacker Factor Solut. 6(2), 1–2 (2007)

    Google Scholar 

Download references

Acknowledgements

This research is supported by the National Key Research and Development Program of China (2018YFB0804202, 2018YFB0804203), the Regional Joint Fund of NSFC (U19A2057), the National Natural Science Foundation of China (61672259, 61876070), and the Jilin Province Science and Technology Development Plan Project (20190303134SF, 20180201064SF).

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Jilin University, Changchun, 130012, China

    Dengyun Xu, Xuanjing Shen, Yongping Huang & Zenan Shi

  2. Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012, China

    Dengyun Xu, Xuanjing Shen, Yongping Huang & Zenan Shi

Authors
  1. Dengyun Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuanjing Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yongping Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zenan Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zenan Shi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, D., Shen, X., Huang, Y. et al. RB-Net: integrating region and boundary features for image manipulation localization. Multimedia Systems 29, 3055–3067 (2023). https://doi.org/10.1007/s00530-022-00903-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-022-00903-z

Keywords

Search

Navigation