Abstract
Due to the rapid growth of digital multimedia technology, the importance of multimedia data such as digital images/videos is increasing rapidly in diverse domains. This includes surveillance CCTV footages, which act as primary evidences in many court cases, including highly sensitive contexts. Today, with the wide availability of low-cost image/video manipulating software, digital images/videos have become highly vulnerable to manipulation/modification attacks; one such attack is the class of object-based forgery in surveillance videos. In this paper, we propose a Capsule Network based digital forensic technique for detection of object-based forgery in surveillance videos. In the proposed technique, we use motion residual, computed from every video frame, to extract intra- and inter-frame inherent statistical characteristics of the video sequence, as the input of capsule network. Our experimental results indicate that the proposed technique achieves significant performance in terms of authentic, double compressed and forged frame detection, irrespective of the group of pictures length and degree of compression in videos.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.References
Aghamaleki JA, Behrad A (2016) Inter-frame video forgery detection and localization using intrinsic effects of double compression on quantization errors of video coding. Signal Process Image Commun 47:289–302
Amerini I, Becarelli R, Caldelli R, Del Mastio A (2014) Splicing forgeries localization through the use of first digit features. In: IEEE international workshop on information forensics and security (WIFS), pp 143–148
Amerini I, Galteri L, Caldelli R, Del Bimbo A (2019) Deepfake video detection through optical flow based CNN. In: IEEE/CVF international conference on computer vision workshop (ICCVW), pp 1205–1207
Bakas J, Naskar R (2018) A digital forensic technique for inter-frame video forgery detection based on 3D CNN. In: International conference on information systems security, (ICISS 2018). Springer, pp 304–317
Bhartiya G, Jalal AS (2017) Forgery detection using feature-clustering in recompressed JPEG images. Multim Tools Appl 76(20):20799–20814
Castiglione A, Cattaneo G, De Santis A (2011) A forensic analysis of images on online social networks. In: Third international conference on intelligent networking and collaborative systems, IEEE, pp 679–684
Chen S, Tan S, Li B, Huang J (2016) Automatic detection of object-based forgery in advanced video. IEEE Trans Circuits Syst Video Technol 26(11):2138–2151
D’Amiano L, Cozzolino D, Poggi G, Verdoliva L (2018) A patchmatch-based dense-field algorithm for video copy-move detection and localization. IEEE Trans Circuits Syst Video Technol 29(3):669–682
Fadl S, Han Q, Li Q (2019) Surveillance video authentication using universal image quality index of temporal average. In: International workshop on digital watermarking (IWDW 2018). Springer, pp 337–350
Fadl SM, Han Q, Li Q (2018) Inter-frame forgery detection based on differential energy of residue. IET Image Process 13(3):522–528
Gan Y, Yang J, Lai W (2019) Video object forgery detection algorithm based on VGG-11 convolutional neural network. In: International conference on intelligent computing. Automation and systems (ICICAS), IEEE, pp 575–580
Gonzalez RC, Woods RE, Masters BR (2008) Digital image processing third edition. Pearson Prentice Hall, New Jersey, pp 743–747
Gull S, Loan NA, Parah SA, Sheikh JA, Bhat GM (2020) An efficient watermarking technique for tamper detection and localization of medical images. J Ambient Intell Humaniz Comput 11(5):1799–1808
Hinton GE, Krizhevsky A, Wang SD (2011) Transforming auto-encoders. In: International conference on artificial neural networks. Springer, pp 44–51
Hinton GE, Sabour S, Frosst N (2018) Matrix capsules with EM routing. In: International conference on learning representations
Katebi R, Zhou Y, Chornock R, Bunescu R (2019) Galaxy morphology prediction using capsule networks. Mon Not R Astron Soc 486(2):1539–1547
Kim M, Chi S (2019) Detection of centerline crossing in abnormal driving using CapsNet. J Supercomput 75(1):189–196
Kodovskỳ J, Fridrich J (2009) Calibration revisited. In: Proceedings of the 11th ACM workshop on multimedia and security, pp 63–74
Kodovsky J, Fridrich J (2012) Steganalysis of JPEG images using rich models. In: Media watermarking, security, and forensics, International Society for Optics and Photonics, vol 8303, p 83030A
Kodovsky J, Fridrich J, Holub V (2011) Ensemble classifiers for steganalysis of digital media. IEEE Trans Inf Forensics Secur 7(2):432–444
Labartino D, Bianchi T, De Rosa A, Fontani M, Vázquez-Padín D, Piva A, Barni M (2013) Localization of forgeries in MPEG-2 video through GOP size and DQ analysis. In: IEEE 15th international workshop on multimedia signal processing (MMSP), IEEE, pp 494–499
Li Y, Zhou J (2019) Fast and effective image copy-move forgery detection via hierarchical feature point matching. IEEE Trans Inf Forensics Secur 14(5):1307–1322
Lin CS, Tsay JJ (2014) A passive approach for effective detection and localization of region-level video forgery with spatio-temporal coherence analysis. Dig Investig 11(2):120–140
Lin PY (2009) Basic image compression algorithm and introduction to JPEG standard. National Taiwan University, Taipei
Liu Y, Huang T (2017) Exposing video inter-frame forgery by Zernike opponent chromaticity moments and coarseness analysis. Multim Syst 23(2):223–238
Long C, Smith E, Basharat A, Hoogs A (2017) A C3D-based convolutional neural network for frame dropping detection in a single video shot. In: IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp 1898–1906
Mohanarathinam A, Kamalraj S, Venkatesan GP, Ravi RV, Manikandababu C (2020) Digital watermarking techniques for image security: a review. J Ambient Intell Humaniz Comput 11(8):3221–3229
Nguyen HH, Tieu TND, Nguyen-Son HQ, Nozick V, Yamagishi J, Echizen I (2018) Modular convolutional neural network for discriminating between computer-generated images and photographic images. In: Proceedings of the 13th international conference on availability, reliability and security, pp 1–10
Nguyen HH, Yamagishi J, Echizen I (2019) Capsule-forensics: using capsule networks to detect forged images and videos. In: IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 2307–2311
Nie Y, Ma KK (2002) Adaptive rood pattern search for fast block-matching motion estimation. IEEE Trans Image Process 11(12):1442–1449
Pandey RC, Singh SK, Shukla KK (2017) A passive forensic method for video: exposing dynamic object removal and frame duplication in the digital video using sensor noise features. J Intell Fuzzy Syst 32(5):3339–3353
Pizzolante R, Castiglione A, Carpentieri B, De Santis A, Castiglione A (2014) Protection of microscopy images through digital watermarking techniques. In: International conference on intelligent networking and collaborative systems, IEEE, pp 65–72
Poncelet J, Renkens V, Van hamme H, (2021) Low resource end-to-end spoken language understanding with capsule networks. Comput Speech Lang 66:101142
Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. In: Advances in neural information processing systems (NIPS 2017), pp 3856–3866
Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: 3rd international conference on learning representations (ICLR2015)
Sitara K, Mehtre B (2016) Digital video tampering detection: an overview of passive techniques. Dig Investig 18(Supplement C):8–22
Su K, Kundur D, Hatzinakos D (2005) Statistical invisibility for collusion-resistant digital video watermarking. IEEE Trans Multim 7(1):43–51
Su L, Huang T, Yang J (2015) A video forgery detection algorithm based on compressive sensing. Multim Tools Appl 74(17):6641–6656
Su L, Luo H, Wang S (2019) A novel forgery detection algorithm for video foreground removal. IEEE Access 7:109719–109728
Vazquez-Padin D, Fontani M, Bianchi T, Comesana P, Piva A, Barni M (2012) Detection of video double encoding with GOP size estimation. In: IEEE international workshop on information forensics and security (WIFS), pp 151–156
Yu L, Wang H, Han Q, Niu X, Yiu SM, Fang J, Wang Z (2016) Exposing frame deletion by detecting abrupt changes in video streams. Neurocomputing 205:84–91
Acknowledgements
This research is partially funded by the following projects:(1) Department of Science and Technology (DST), Govt. of India, Grant No. DST/ICPS/Cluster/CS Research/2018 (General) dated: 13/03/2019. (2) Project titled “Deep learning applications for computer vision task” funded by National Institute of Technology Rourkela Overseas Alumni Association (NITROAA) with support of Lenovo P920 workstation and NVIDIA Corporation with support of NVIDIA Titan V and Quadro RTX 8000 GPUs.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Bakas, J., Naskar, R., Nappi, M. et al. Object-based forgery detection in surveillance video using capsule network. J Ambient Intell Human Comput 14, 3781–3791 (2023). https://doi.org/10.1007/s12652-021-03511-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-021-03511-3