Skip to main content
SpringerLink
Log in

Content based video retrieval using dynamic textures

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In recent days, design of smart city applications is attracting several researchers offering improvised services to citizens through efficient management of day to day life activities such as business, safety, public utility, transportation and hospitality etc. In most of the smart city applications video retrieval plays crucial role. Through implementation of video retrieval we can achieve several smart city innovates such as monitoring of traffic or crowd etc. The existing video retrieval system depends on either ontological concepts or training based concepts. Implementation of ontological concepts has shown drawbacks such as miss-assignment of tags to the database videos leads to poor efficiency and also needs huge manual effort for the purpose of onnotation. Training based concepts needs prior knowledge and also takes more time for the purpose of training and to overcome all these drawbacks content based video retrieval systems (CBVR) has been evolved. In most of the existing CBVR systems the principal challenge is semantic gap between the user defined rich semantics and the system defined low level features of the scene. In the present article, we propose an algorithm of CBVR using dynamic textures. Statistical textures with the inclusion of motion of the pattern, change in illumination of the pattern, intrinsic change to the pattern becomes dynamic textures. Dynamic textures are best suited to videos containing moving objects. In this article, LBP-TOP a variant of dynamic texture has been used as a feature for video retrieval. LBP-TOP has the capability of jointly describing motion and appearance features. The LBP-TOP features are invariant to illumination, rotation and local translation. These significant benefits support the use of LBP-TOP in the proposed method. The proposed method uses query video clip, which consists randomly selected ten example frames. The proposed CBVR has three stages offline processing, online processing, a matching & retrieval stage. In offline processing, we extract keyframes of database videos using Pearson Correlation Coefficient (PCC) and Color Moments (CM) and then LBP-TOP feature of keyframes have been extracted and used to represent entire database video. In online processing, we extract LBP-TOP features of query video and then these features will be given as input to matching & retrieval stage where, we calculate euclidean distance between LBP-TOP features of database keyframes and query video frames to retrieve videos with less distance. To prove effectiveness of the proposed method it have been tested on 108 videos of standard traffic dataset which is available publicly and compared with the other state-of-the-art methods, both qualitatively and quantitavely. Quantitative performance evaluation has been carried out using the evaluation parameters: Precision, Recall, Jaccard Index, Accuracy, Specificity and E-measure. Both qualitative and quantitative performance show that the proposed method performed well than the other state-of-the-art methods, and success of the proposed method lies under incorporation of dynamic textures. The proposed algorithm can be used in real-time applications like traffic monitoring. The proposed CBVR system can be used to monitor traffic through feature matching between query scene and the database video. If the query is matched with low traffic video of database then the algorithm displays output as low traffic time. In similar manner, medium traffic or heavy traffic times will be detected by the algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Agarwal R, Verma OP (2019) An efficient copy move forgery detection using deep learning feature extraction and matching algorithm. Multimed Tools Appl 23:1–22

    Google Scholar 

  2. Aote SS, Potnurwar A (2019) An automatic video annotation framework based on two level keyframe extraction mechanism. Multimed Tools Appl 78(11):14465–14484

    Article  Google Scholar 

  3. Araujo A, Girod B (2017) Large-scale video retrieval using image queries. IEEE Trans Circuits Systems Vid Technol 28(6):1406–1420

    Article  Google Scholar 

  4. Bommisetty RM, Prakash O, Khare A (2020) Keyframe extraction using Pearson correlation coefficient and color moments. Multimedia Systems 26:267–299. https://doi.org/10.1007/s00530-019-00642-8

  5. Bommisetty RM, Khare A, Khare M, Palanisamy P (2021) Content-based video retrieval using integration of Curvelet transform and simple linear iterative clustering. Int J Image Graph 16:2250018

    Google Scholar 

  6. Mounika Bommisetty R, Khare A, Siddiqui TJ et al (2021) Fusion of gradient and feature similarity for Keyframe extraction. Multimed Tools Appl 80:15429–15467. https://doi.org/10.1007/s11042-020-10390-x

  7. Chatzigiorgaki M, Skodras AN (2009) Real-time keyframe extraction towards video content identification. In2009 16th international conference on digital signal processing 2009 Jul 5 (pp. 1-6). IEEE

  8. Dave N, Holia M (2020) News story retrieval based on textual query. Int J Eng Adv Technol (IJEAT) ISSN: 2249–8958, Volume-9 Issue-3. https://doi.org/10.35940/ijeat.C5264.029320, https://www.scopus.com/sourceid/21100899502

  9. Fleischman M, Evans H, Roy D (2007) Unsupervised content-based indexing for sports video retrieval. In Proceedings of the 15th ACM international conference on Multimedia, pp.473–474. https://doi.org/10.1145/1291233.1291347

  10. Gornale SS, Babaleshwar AK, Yannawar PL (2019) Analysis and Detection of Content based video retrieval. Int J Image Graph Signal Process, 11(3):43–57. https://doi.org/10.5815/ijigsp.2019.03.06

  11. Gu L, Liu J, Qu A (2017) Performance evaluation and scheme selection of shot boundary detection and keyframe extraction in content based video retrieval. Int J Digit Crime Forensics (IJDCF) 9(4): 15–29. https://doi.org/10.4018/IJDCF.2017100102

  12. Hou S, Zhou S (2013) Audio-visual-based query by example video retrieval. Math Probl Eng 2013:1–8

    Article  Google Scholar 

  13. Ashish Khare, B. Reddy Mounika, Manish Khare (2020) Keyframe extraction using binary robust invariant scalable keypoint features. In Proc. SPIE 11433, Twelfth International Conference on Machine Vision (ICMV 2019), 1143308. https://doi.org/10.1117/12.2559105

  14. Lee D, Lee GG (2008) A korean spoken document retrieval system for lecture search. CIP GEGEVENS KONINKLIJKE BIBLIOTHEEK, DEN HAAG : 73

  15. Li X, Zhou F, Xu C, Ji J, Yang G (2020) Sea: Sentence encoder assembly for video retrieval by textual queries. IEEE Trans Multimed

  16. Liu Y, Kong D, Zhao D, Gong X, Han G (2018) A Point cloud registration algorithm based on feature extraction and matching. Math Probl Eng 2018:7352691. https://doi.org/10.1155/2018/7352691

  17. Loukas C, Varytimidis C, Rapantzikos K, Kanakis MA (2018) Keyframe extraction from laparoscopic videos based on visual saliency detection. Comput Methods Prog Biomed 165:13–23

    Article  Google Scholar 

  18. Lu G, Zhou Y, Li X et al (2017) Unsupervised, efficient and scalable key-frame selection for automatic summarization of surveillance videos. Multimed Tools Appl 76:6309–6331. https://doi.org/10.1007/s11042-016-3263-z

  19. Memar S, Affendey LS, Mustapha N, Doraisamy SC, Ektefa M (2013) An integrated semantic-based approach in concept based video retrieval. Multimed Tools Appl 64(1):77–95

    Article  Google Scholar 

  20. Money AG, Agius H (2010) Elvis: entertainment-led video summaries. ACM Trans Multimed Comput Commun Appl (TOMM) 6(3):1–30

    Article  Google Scholar 

  21. Mounika BR, Khare A (n.d.) CBVR using histogram of gradients and frame fusion. In twelfth international conference on machine vision (ICMV 2019) 2020 Jan 31 (Vol. 11433, p. 114332J). Int Soc Optics Photonics

  22. Mühling M, Korfhage N, Müller E, Otto C, Springstein M, Langelage T, Veith U, Ewerth R, Freisleben B (2017) Deep learning for content-based video retrieval in film and television production. Multimed Tools Appl 76(21):22169–22194

    Article  Google Scholar 

  23. MühlingM,MeisterM, Korfhage N,Wehling J, Hörth A, Ewerth R, Freisleben B (2019) Content-based video retrieval in historical collections of the German broadcasting archive. Int J Digit Libr 20(2):167–183. https://doi.org/10.48550/arXiv.1702.03790

  24. Nandini HM, Chethan HK, Rashmi BS (2020) Shot based keyframe extraction using edge-LBP approach. J King Saud Univ-Computer Inf Sci

  25. Page KR, Lewis D, Weigl DM (2019) MELD: a linked data framework for multimedia access to music digital libraries. In2019 ACM/IEEE joint conference on digital libraries (JCDL) 2019 Jun 2 (pp 434-435). IEEE

  26. Péteri R, Fazekas S, Huiskes MJ (2010) DynTex: a comprehensive database of dynamic textures. Pattern Recogn Lett 31(12):1627–1632

    Article  Google Scholar 

  27. Rui Y et al (2004) A unified framework for video summarization, browsing and retrieval. Mitsubishi Elect Res Lab Tech Rep

  28. Sandeep R, Sharma S, Thakur M, Bora PK (2016) Perceptual video hashing with application to indexing and retrieval of near-identical videos. Multimed Tools Appl 75(13):7779–7797

    Article  Google Scholar 

  29. Shekar BH, Uma KP, Raghurama Holla K (2016) Video clip retrieval based on LBP variance. Procedia Comput Sci 89:828–835

    Article  Google Scholar 

  30. Shi Y, Wei Z, Ling H, Wang Z, Shen J, Li P (2020) Person retrieval in surveillance videos via deep attribute mining and reasoning. IEEE Trans Multimed 2

  31. Song J, Gao L, Liu L, Zhu X, Sebe N (2018) Quantization-based hashing: a general framework for scalable image and video retrieval. Pattern Recogn 75:175–187

    Article  Google Scholar 

  32. Thomas SS, Gupta S, Venkatesh KS (2017) Perceptual synoptic view-based video retrieval using metadata. Sign Image Vid Process 11(3):549–555

    Article  Google Scholar 

  33. Wang P, Sun X, Diao W, Fu K (2019) FMSSD: feature-merged single-shot detection for multiscale objects in large-scale remote sensing imagery. IEEE Trans Geosci Remote Sens 58(5):3377–3390

    Article  Google Scholar 

  34. Wu G, Han J, Guo Y, Liu L, Ding G, Ni Q, Shao L (2018) Unsupervised deep video hashing via balanced code for large-scale video retrieval. IEEE Trans Image Process 28(4):1993–2007

    Article  MathSciNet  Google Scholar 

  35. Yang H, Meinel C (2014) Content based lecture video retrieval using speech and video text information. IEEE Trans Learn Technol 7(2):142–154. https://doi.org/10.1109/TLT.2014.2307305

  36. Yang Y, Lovell BC, Dadgostar F (2009) Content-Based Video Retrieval (CBVR) system for CCTV surveillancevideos. 2009 digital image computing: techniques and applications, pp. 183–187. https://doi.org/10.1109/DICTA.2009.36

  37. Yue H (2018) Unstructured healthcare data archiving and retrieval using Hadoop and Drill. Int J Big Data Anal Healthcare (IJBDAH) 3(2):28–44

    Article  Google Scholar 

  38. Zhang L, Liu Y, Zhang J (2019) Saliency detection based on self-adaptive multiple feature fusion for remote sensing images. Int J Remote Sens 40(22):8270–8297

    Article  Google Scholar 

  39. Zhang C, Lin Y, Zhu L, Liu A, Zhang Z, Huang F (2019) CNN-VWII: an efficient approach for large-scale video retrieval by image queries. Pattern Recogn Lett 123:82–88

    Article  Google Scholar 

  40. Zhao G, Pietikainen M (2006) Local binary pattern descriptors for dynamic texture recognition. 18th International Conference on Pattern Recognition (ICPR'06). Vol. 2. IEEE

  41. Zhao G, Pietikainen M (2007) Dynamic texture recognition using local binarypatterns with an application to facial expressions. IEEE Trans Pattern Anal Mach Intell 29(6):915–928. https://doi.org/10.1109/TPAMI.2007.1110

  42. Zhou J, Kwan C (2018) Anomaly detection in low quality traffic monitoring videos using optical flow. In Pattern Recognition and Tracking XXIX 2018 Apr 30 (Vol. 10649, p. 106490F). International Society for Optics and Photonics. Ma S, Chen X, Li Z, Yang Y. A retrieval optimized surveillance video storage system for campus application scenarios. J Elect Comput Eng

  43. Zhou Z, Chen J, Yang, C-N Sun X (2019) Video copy detection using spatio-temporal CNN features. IEEE Access 7:100658100665. https://doi.org/10.1109/ACCESS.2019.2930173

  44. Zong Z, Gong Q (2017) Key frame extraction based on dynamic color histogram and fast wavelet histogram. In2017 IEEE international conference on information and automation (ICIA) 2017 Jul 18 (pp. 183-188). IEEE

Download references

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, National Institute of Technology, Tiruchirappalli, India

    B. Reddy Mounika, P Palanisamy & Hotta Himanshu Sekhar

  2. Department of Electronics and Communication Engineering, University of Allahabad, Prayagraj, India

    Ashish Khare

Authors
  1. B. Reddy Mounika
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P Palanisamy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hotta Himanshu Sekhar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ashish Khare
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to P Palanisamy.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mounika, B.R., Palanisamy, P., Sekhar, H.H. et al. Content based video retrieval using dynamic textures. Multimed Tools Appl 82, 59–90 (2023). https://doi.org/10.1007/s11042-022-13086-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13086-6

Keywords

Search

Navigation