Skip to main content
SpringerLink
Log in

Detection of motion compensated frame interpolation via motion-aligned temporal difference

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Motion compensated frame interpolation (MCFI) is a special frame based video manipulation, which increases the temporal continuity of low frame rate videos by synthesizing new frames between successive frames. MCFI might also be used to counterfeit high frame-rate videos, which mislead users’ attraction and waste storage spaces in video-sharing websites. Existing MCFI detectors are designed to judge the absence or presence of MCFI forgery in a controllable environment of known MCFI techniques. Practical detector should consider the possibility of unknown MCFI techniques. We are motivated to propose a robust MCFI detector for more practical scenarios. Considering the effects of non-motion regions in candidate frame, the statistical moments are firstly extracted from motion-aligned frame differences (MAFD). Then, the one-class support vector machine (SVM), following a training stage capturing the properties of original frames, is exploited to judge whether the candidate frame is interpolated by MCFI or not. Finally, a special interpolated frame detection (SIFD) is designed to pick out interpolated frames, which are synthesized from two consecutive reference frames with no motion vectors (MVs) or less MVs. A series of experiments evaluated on four representative MCFI techniques have shown promising results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. Available on http://iphome.hhi.de/suehring/tml/

  2. They are Bowing, Bus, Coastguard, Container, Flower, Football, Highway, Mobile, Silent, Tempete, Waterfall

References

  1. Available on the online video Databases. http://media.xiph.org/video/derf/#

  2. Available on the online video Databases. http://www.nada.kth.se/cvap/actions/

  3. Bestagini P, Battaglia S, Milani S, Tagliasacchi M, Tubaro S (2013) Detection of temporal interpolation in video sequences. In: Proceedings of the International Conference Acoustic, Speech Signal Process, pp 3033–3037

  4. Bian S, Luo W, Huang J (2014) Detecting video frame-rate upconversion based on periodic properties of inter-frame similarity. Multimed Tools Appl 72(1):437–451

    Article  Google Scholar 

  5. Black MJ, Anandan P (1993) A framework for the robust estimation of optical flow. In: Proceedings of the 4th International Conference on Computing Vision, pp 231–236

  6. Boser BE, Guyon IM, Vapnik VN (1992) A training algorithm for optimal margin classifiers. In: Proceedings of the ACM 5th Annual Workshop COLT, Pittsburgh, PA USA, pp 144–152

  7. Chang X, Ma Z, Lin M et al (2017) Feature interaction augmented sparse learning for fast kinect motion detection. IEEE Trans Image Process 26(8):3911–3920

    Article  MathSciNet  Google Scholar 

  8. Chang X, Ma Z, Yang Y et al (2017) Bi-level semantic representation analysis for multimedia event detection. IEEE Trans Cybern 47(5):1180–1197

    Article  Google Scholar 

  9. Chang X, Yang Y (2017) Semisupervised feature analysis by mining correlations among multiple tasks. IEEE Trans Neural Netw Learn Syst 28(10):2294–2305

    Article  MathSciNet  Google Scholar 

  10. Chang X, Yu YL, Yang Y et al (2017) Semantic pooling for complex event analysis in untrimmed videos. IEEE Trans Pattern Anal Mach Intell 39(8):1617–1632

    Article  Google Scholar 

  11. Chao J, Jiang X, Sun T (2012) A novel video inter-frame forgery model detection scheme based on optical flow consistency. In: Digital forensics and watermaking. Springer, Berlin, pp 267–281

    Chapter  Google Scholar 

  12. Chen R, Yang G, Zhu N (2014) Detection of object-based manipulation by the statistical features of object contour. Forensic Sci Int 236:164–169

    Article  Google Scholar 

  13. Chen S, Tan S, Li B, Huang J (2016) Automatic detection of object-based forgery in advanced video. IEEE Trans Circuits Syst Video Technol 26(11):2138–2151

    Article  Google Scholar 

  14. Conotter V, OBrien J, Farid H (2012) Exposing digital forgeries in ballistic motion. IEEE Trans Inf Forensics Secur 7(1):283–296

    Article  Google Scholar 

  15. Dar Y, Bruckstein AM (2015) Motion-compensated coding and frame rate up-conversion: models and analysis. IEEE Trans Image Process 24(7):2051–2066

    MathSciNet  MATH  Google Scholar 

  16. Ding X, Yang G, Li R, Zhang L, Li Y, Sun X (2017) Identification of motion-compensated frame rate up-conversion based on residual signal. IEEE Trans Circuits Syst Video Technol. https://doi.org/10.1109/TCSVT.2017.2676162

    Article  Google Scholar 

  17. Kang SJ, Cho SI, Yoo S, Kim YH (2012) Scene change detection using multiple histograms for motion-compensated frame rate upconversion. J Display Technol 8(3):121–126

    Article  Google Scholar 

  18. Kodovsky J, Fridrich J, Holub V (2012) Ensemble classifiers for steganalysis of digital media. IEEE Trans Inf Forensics Secur 7(2):432–444

    Article  Google Scholar 

  19. Lee WH, Choi K, Ra JB (2014) Frame rate up conversion based on variational image fusion. IEEE Trans Image Process 23(1):399–412

    Article  MathSciNet  Google Scholar 

  20. Li H, Luo W, Qiu X, Huang J (2016) Identification of various image operations using residual-based features. IEEE Trans Circuits Syst Video Technol. https://doi.org/10.1109/TCSVT.2016.2599849

    Article  Google Scholar 

  21. Li R, Gan Z, Cui Z, Tang G, Zhu X (2014) Multi-channel mixed-pattern based frame rate up-conversion using spatio-temporal motion vector refinement and dual weighted overlapped block motion compensation. J Disp Technol 10(12):1010–1023

    Article  Google Scholar 

  22. Li R, Liu Z, Zhang Y, Li Y, Fu Z (2018) Noise-level estimation based detection of motion-compensated frame interpolation in video sequences. Multimed Tools Appl 77(1):663–688

    Article  Google Scholar 

  23. Li Z, Nie F, Chang X, Yang Y (2017) Beyond trace ratio: weighted harmonic mean of trace ratios for multiclass discriminant analysis. IEEE Trans Knowl Data Eng 29(10):2100–2110

    Article  Google Scholar 

  24. Liu HB, Xin RQ, Zhao DB, Ma SW, Gao W (2012) Multiple hypotheses bayesian frame rate up-conversion by adaptive fusion of motion-compensated interpolations. IEEE Trans Circuits Syst Video Technol 22(8):1188–1198

    Article  Google Scholar 

  25. Liu Y, Huang T (2015) Exposing video inter-frame forgery by Zernike opponent chromaticity moments and coarseness analysis. Multimed Syst 23(2):223–238

    Article  MathSciNet  Google Scholar 

  26. Milani S, Fontani M, Bestagini P, Barni M, Piva A, Tagliasacchi M, Tubaro S (2012) An overview on video forensics. APSIPA Trans Signal Inf Process 1:1–18

    Article  Google Scholar 

  27. Nie Y, Ma KK (2002) Adaptive rood pattern search for fast block-matching motion estimation. IEEE Trans Image Process 11(12):1442–1448

    Article  Google Scholar 

  28. Po LM, Ma WC (1996) A novel four-step search algorithm for fast block motion estimation. IEEE Trans Circuits Syst Video Technol 6(3):313–317

    Article  Google Scholar 

  29. Rijsbergen CJV (1979) Information retrieval. Butterworth-Heinemann, Newton

    MATH  Google Scholar 

  30. Rocha A, Scheirer W, Boult TE, Goldenstein S (2011) Vision of the unseen: current trends and challenges in digital image and video forensics. ACM Comput Surveys 43:1–42

    Article  Google Scholar 

  31. Stamm MC, Lin W, Liu KJ (2012) Temporal forensics and anti-forensics for motion compensated video. IEEE Trans Inf Forensics Security 7(4):1315–1329

    Article  Google Scholar 

  32. Stamm MC, Wu M, Liu KJ (2013) Information forensics: an overview of the first decade. IEEE Access 1:167–200

    Article  Google Scholar 

  33. Vapnik VN, Lerner A (1963) Pattern recognition using generalized portrait method. Autom Remote Control 24(6):774–780

    Google Scholar 

  34. Wang T, Snoussi H (2014) Detection of abnormal visual events via global optical flow orientation histogram. IEEE Trans Inf Forensics Secur 9(6):988–998

    Article  Google Scholar 

  35. Wang W, Farid H (2007) Exposing digital forgeries in interlaced and deinterlaced video. IEEE Trans Inf Forensics Security 2(3):438–449

    Article  Google Scholar 

  36. Wu Y, Jiang X, Sun T, Wang W (2014) Exposing video inter-frame forgery based on velocity field consistency. In: Proceedings of the International Conference Acoustic, Speech Signal Process, pp 2674–2678

  37. Xia M, Yang G, Li L, Li R, Sun X (2016) Detecting video frame rate up-conversion based on frame-level analysis of average texture variation. Multimed Tools Appl:1–23

  38. Yao Y, Yang G, Sun X, Li L (2016) Detecting video frame-rate up-conversion based on periodic properties of edge-intensity. Journal of Information Security and Applications 26:39–50

    Article  Google Scholar 

  39. Yoo DG, Kang SJ, Kim YH (2013) Direction-select motion estimation for motion-compensated frame rate up-conversion. J Display Technol 9(10):840–850

    Article  Google Scholar 

  40. Zhang J, Su Y, Zhang M (2009) Exposing digital video forgery by ghost shadow artifact. In: Proceedings of the 1st ACM workshop on multimedia in forensics,(mifor09), pp 49–54

  41. Zhu S, Ma KK (2000) A new diamond search algorithm for fast block-matching motion estimation. IEEE Trans Image Process 9(2):287–290

    Article  Google Scholar 

Download references

Acknowledgements

The authors appreciate Dr. Ran Li in Xinyang Normal University, China for permission to use their codes in theirs experiments. They appreciate Dr. Won Hee Lee in Korea Advanced Institute of Science and Technology, Korea for providing up-converted videos in the experiments. They also appreciate Dr Ningbo Zhu in Hunan University, China for providing technical editing of the revised manuscript. This work is supported in part by the National Natural Science Foundation of China (61572183, 61379143).

Author information

Authors and Affiliations

  1. College of Computer Science and Electronic Engineering, Hunan University, Changsha, 410082, China

    Xiangling Ding, Yue Li, Ming Xia, Jiale He & Gaobo Yang

  2. School of Information Science and Engineering, Jishou University, Jishou, Hunan, 416000, China

    Xiangling Ding

Authors
  1. Xiangling Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yue Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ming Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiale He
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gaobo Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gaobo Yang.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ding, X., Li, Y., Xia, M. et al. Detection of motion compensated frame interpolation via motion-aligned temporal difference. Multimed Tools Appl 78, 7453–7477 (2019). https://doi.org/10.1007/s11042-018-6504-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6504-5

Keywords

Search

Navigation