Skip to main content
SpringerLink
Log in

Towards general object-based video forgery detection via dual-stream networks and depth information embedding

  • 1190: Depth-Related Processing and Applications in Visual Systems
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The object-based video forgery detection aims to expose tampered regions from video sequences without any codec information. However, existing methods mainly focus on manually selected features and models for a specific task, either splicing or copy-move, while the general representation ability of deep learning models and the correlation of different forensic features have not been fully explored. In this letter, we propose a dual-stream framework to jointly discover and integrate effective features for object-based video forgery detection. First, two different types of branches are employed to extract discriminative features. Then, after the dual-stream feature fusion, a Conditional Random Field (CRF) layer is utilized to further refine segmentation results. Finally, we consider temporal consistency by incorporating the video tracking strategy. Depth information is adopted to refine the localization results. Extensive experiments on four datasets show that the proposed method achieves competitive performance against the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. http://www.grip.unina.it/download/prog/ForgedVideosDataset/

  2. https://www.youtube.com/channel/UCZuuu-iyZvPptbIUHT9tMrA

  3. https://sites.google.com/site/rewindpolimi/downloads/datasets/

  4. http://sulfa.cs.surrey.ac.uk/forged.php

  5. As suggested in [2], object removal is viewed as a special case of copy-move operation for brevity.

References

  1. Al-Qershi OM, Khoo BE (2013) Passive detection of copy-move forgery in digital images: State-of-the-art. Forensic Sci Int 231(1):284–295. https://doi.org/10.1016/j.forsciint.2013.05.027

    Article  Google Scholar 

  2. Al-Sanjary OI, Ahmed AA, Sulong G (2016) Development of a video tampering dataset for forensic investigation. Forensic Sci Int 266:565–572. https://doi.org/10.1016/j.forsciint.2016.07.013

    Article  Google Scholar 

  3. Aloraini M, Sharifzadeh M, Schonfeld D (2020) Sequential and patch analyses for object removal video forgery detection and localization. IEEE Trans Circuits Syst Video Technol, pp 1–1. https://doi.org/10.1109/TCSVT.2020.2993004

  4. Bappy JH, Simons C, Nataraj L, Manjunath B, Roy-Chowdhury AK (2019) Hybrid LSTM and encoder-decoder architecture for detection of image forgeries. IEEE Trans Image Process 28(7):3286–3300

    Article  MathSciNet  Google Scholar 

  5. Bayar B, Stamm MC (2018) Constrained convolutional neural networks: a new approach towards general purpose image manipulation detection. IEEE Trans Inform Forensics Secur 13(11):2691–2706. https://doi.org/10.1109/TIFS.2018.2825953

    Article  Google Scholar 

  6. Bestagini P, Milani S, Tagliasacchi M, Tubaro S (2013) Local tampering detection in video sequences. In: Proceedings of the IEEE International Workshop on Multimedia Signal Processing (MMSP), pp 488–493

  7. Caelles S, Maninis KK, Pont-Tuset J, Leal-Taixé L, Cremers D, Van Gool L (2017) One-shot video object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  8. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  9. Chen S, Tan S, Li B, Huang J (2016) Automatic detection of object-based forgery in advanced video. IEEE Trans Circuits Syst Video Technol 26 (11):2138–2151

    Article  Google Scholar 

  10. Cong R, Lei J, Fu H, Hou J, Huang Q, Kwong S (2020) Going from RGB to RGBD saliency: a Depth-Guided transformation model. IEEE Trans Cybern 50(8):3627–3639. https://doi.org/10.1109/TCYB.2019.2932005

    Article  Google Scholar 

  11. Cong R, Lei J, Fu H, Huang Q, Cao X, Ling N (2019) HSCS: Hierarchical sparsity based co-saliency detection For RGBD images. IEEE Trans Multimedia 21(7):1660–1671

    Article  Google Scholar 

  12. Cozzolino D, Verdoliva L (2020) Noiseprint: A CNN-based Camera Model Fingerprint. IEEE Trans Inform Forensics Secur 15:144–159. https://doi.org/10.1109/TIFS.2019.2916364

    Article  Google Scholar 

  13. Cozzolino Giovanni Poggi Luisa Verdoliva D (2019) Extracting camera-based fingerprints for video forensics. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 130–137

  14. D’Amiano L, Cozzolino D, Poggi G, Verdoliva L (2018) A patchmatch-based dense-field algorithm for video copy-move detection and localization. IEEE Trans Circuits Syst Video Technol 29(3):669–682

    Article  Google Scholar 

  15. D’Avino D, Cozzolino D, Poggi G, Verdoliva L (2017) Autoencoder with recurrent neural networks for video forgery detection. Electron Imaging 2017(7):92–99. https://doi.org/10.2352/ISSN.2470-1173.2017.7.MWSF-330

    Article  Google Scholar 

  16. D’Avino D, Cozzolino D, Poggi G, Verdoliva L (2017) Autoencoder with recurrent neural networks for video forgery detection. Electron Imaging 2017(7):92–99

    Article  Google Scholar 

  17. Dua S, Singh J, Parthasarathy H (2020) Detection and localization of forgery using statistics of DCT and Fourier components. Signal Process Image Commun 82(115):778. https://doi.org/10.1016/j.image.2020.115778

    Article  Google Scholar 

  18. Farid H (2019) Image forensics. Ann Rev Vis Sci 5(1):549–573. https://doi.org/10.1146/annurev-vision-091718-014827

    Article  Google Scholar 

  19. Huh M, Liu A, Owens A, Efros AA (2018) Fighting fake news: Image splice detection via learned self-consistency. In: Proceedings of the European Conference on Computer Vision, pp 101–117

  20. Johnston P, Elyan E (2019) A review of digital video tampering: from simple editing to full synthesis. Digit Investig 29:67–81. https://doi.org/10.1016/j.diin.2019.03.006

    Article  Google Scholar 

  21. Kohli A, Gupta A, Singhal D (2020) CNN Based localisation of forged region in object-based forgery for HD videos. IET Image Process 14(5):947–958

    Article  Google Scholar 

  22. Li C, Cong R, Kwong S, Hou J, Fu H, Zhu G, Zhang D, Huang Q (2020) ASIF-Net: Attention Steered Interweave Fusion Network for RGB-d Salient Object Detection. IEEE Transactions on Cybernetics, pp 1–13

  23. Li C, Cong R, Piao Y, Xu Q, Loy CC (2020) RGB-D salient object detection with cross-modality modulation and selection. In: Proceedings of the European Conference on Computer Vision

  24. Li H, Huang J (2019) Localization of deep inpainting using high-pass fully convolutional network. In: Proceedings of the IEEE International Conference on Computer Vision, pp 8301–8310

  25. Lin CS, Tsay JJ (2014) A passive approach for effective detection and localization of region-level video forgery with spatio-temporal coherence analysis. Digit Investig 11(2):120–140. https://doi.org/10.1016/j.diin.2014.03.016

    Article  Google Scholar 

  26. Liu B, Pun CM (2018) Locating splicing forgery by fully convolutional networks and conditional random field. Signal Process Image Commun 66:103–112. https://doi.org/10.1016/j.image.2018.04.011

    Article  Google Scholar 

  27. Liu Y, Zhu X, Zhao X, Cao Y (2019) Adversarial learning for constrained image splicing detection and localization based on atrous convolution. IEEE Trans Inform Forensics and Secur 14(10):2551–2566

    Article  Google Scholar 

  28. Poggi M, Tosi F, Mattoccia S (2018) Learning monocular depth estimation with unsupervised trinocular assumptions. In: Proceedings of the IEEE International Conference on 3D Vision (3DV), IEEE, pp 324–333

  29. Qadir G, Yahaya S, Ho ATS (2012) Surrey University Library for Forensic Analysis (SULFA) of video content. In: Proceedings of the IET Conference on Image Processing, pp 1–6

  30. Rocha A, Scheirer W, Boult T, Goldenstein S (2011) Vision of the unseen: Current trends and challenges in digital image and video forensics. ACM Comput Surv 43(4):1–42. https://doi.org/10.1145/1978802.1978805

    Article  Google Scholar 

  31. Shi X, Chen Z, Wang H, Yeung DY, Wong WK, Woo WC (2015) Convolutional LSTM network: A Machine Learning Approach for Precipitation Nowcasting. In: Advances in neural information processing systems, pp 802–810

  32. Singh RD, Aggarwal N (2018) Video content authentication techniques: a comprehensive survey. Multimedia Syst 24(2):211–240. https://doi.org/10.1007/s00530-017-0538-9

    Article  Google Scholar 

  33. Sitara K, Mehtre B (2016) Digital video tampering detection: an overview of passive techniques. Digit Investig 18:8–22. https://doi.org/10.1016/j.diin.2016.06.003

    Article  Google Scholar 

  34. Su L, Li C, Lai Y, Yang J (2018) A fast forgery detection algorithm based on exponential-fourier moments for video region duplication. IEEE Trans Multimedia 20(4):825–840

    Article  Google Scholar 

  35. Verdoliva L (2020) Media forensics and deepfakes: an overview. IEEE J Sel Top Signal Process 14(5):910–932. https://doi.org/10.1109/JSTSP.2020.3002101

    Article  Google Scholar 

  36. Wang W, Shen J, Porikli F, Yang R (2019) Semi-supervised video object segmentation with super-trajectories. IEEE Trans Pattern Anal Mach Intell 41(4):985–998

    Article  Google Scholar 

  37. Warif NBA, Wahab AWA, Idris MYI, Ramli R, Salleh R, Shamshirband S, Choo KKR (2016) Copy-move forgery detection: Survey, challenges and future directions. J Netw Comput Appl 75:259–278. https://doi.org/10.1016/j.jnca.2016.09.008

    Article  Google Scholar 

  38. Wu Y, Abd-Almageed W, Natarajan P (2019) Mantra-net: Manipulation tracing network for detection and localization of image forgeries with anomalous features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 9543–9552

  39. Zhong JL, Pun CM, Gan YF (2020) Dense moment feature index and best match algorithms for video copy-move forgery detection. Inf Sci 537:184–202. https://doi.org/10.1016/j.ins.2020.05.134

    Article  MathSciNet  Google Scholar 

  40. Zhou P, Han X, Morariu VI, Davis LS (2018) Learning rich features for image manipulation detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1053–1061

  41. Zhu N, Li Z (2018) Blind image splicing detection via noise level function. Signal Process Image Commun 68:181–192. https://doi.org/10.1016/j.image.2018.07.012

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the Science and Technology Planning Project of Tianjin, China (Grant No. 17JCZDJC30700 and 18ZXZNGX00310), the Tianjin Natural Science Foundation (Grant No. 19JCQNJC00300), and the Fundamental Research Funds for the Central Universities of Nankai University (Grant No. 63201192 and 63211116).

Author information

Authors and Affiliations

  1. College of Artificial Intelligence, Nankai University, Tianjin, 300350, China

    Xiao Jin, Zhen He, Jiawei Yu & Jing Xu

  2. Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, Canada

    Yongwei Wang

Authors
  1. Xiao Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhen He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yongwei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiawei Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jing Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Xu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jin, X., He, Z., Wang, Y. et al. Towards general object-based video forgery detection via dual-stream networks and depth information embedding. Multimed Tools Appl 81, 35733–35749 (2022). https://doi.org/10.1007/s11042-021-11126-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11126-1

Keywords

Search

Navigation