Skip to main content
SpringerLink
Log in

Visual significance model based temporal signature for video shot boundary detection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Video shot boundary detection (VSBD) is the fundamental step for video processing algorithms. The goal of any VSBD algorithm is to detect the transitions (abrupt or subtle) in the given video precisely. In this paper, a visual significance model that is suitable for describing transitions in video is introduced. The proposed visual significance model is composed of parameters like color, texture, edge, motion and focus computed over the frames in the video. Once the visually significant region is identified in each frame of the video, the temporal signature is generated through the dissimilarity measure of the visual significance model. The temporal signature is further examined using standard Random Vector Functional Link (RVFL) networks for categorizing the transitions as Abrupt Transitions (AT), Subtle Transitions (ST) and No Transitions (NT). To validate the performance of the proposed visual significance model based VSBD Framework, it is evaluated on benchmarks like VIDEOSEG2004 and TRECVID2001 to detect and categorize the transitions. Comparison of F1-Score measure with prominent early methods reveals that the proposed framework is a promising model for detecting the transitions in videos even in the presence of varying illumination conditions, fast camera and object motion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Algorithm 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Abdulhussain SH, Ramli AR, Mahmmod BM, Saripan MI, al-Haddad SAR, Jassim WA (2019) Shot boundary detection based on orthogonal polynomial. Multimed Tools Appl 78:20361–20382. https://doi.org/10.1007/s11042-019-7364-3

    Article  Google Scholar 

  2. Benoughidene A, Titouna F (2022) A novel method for video shot boundary detection using CNN-LSTM approach. Int J Multimed Inf Retr 11:653–667. https://doi.org/10.1007/s13735-022-00251-8

    Article  Google Scholar 

  3. Bi C, Yuan Y, Zhang J, Shi Y, Xiang Y, Wang Y, Zhang RH (2018) Dynamic mode decomposition based video shot detection. IEEE Access 6:21397–21407. https://doi.org/10.1109/ACCESS.2018.2825106

    Article  Google Scholar 

  4. Chakraborty S, Thounaujam DM, Singh A, Pal G (2022) ALO-SBD: a hybrid shot boundary detection technique for video surveillance system. In: Edge Analytics: Select Proceedings of 26th International Conference—ADCOM 2020. Springer Singapore, Singapore, pp 685–696. https://doi.org/10.1007/978-981-19-0019-8_51

  5. Cyganek B, Woźniak M (2017 Tensor-based shot boundary detection in video streams. New Gener Comput 35(4):311–340. https://doi.org/10.1007/s00354-017-0024-0

  6. Gygli M (2018) Ridiculously fast shot boundary detection with fully convolutional neural networks. In: 2018 International Conference on Content-Based Multimedia Indexing (CBMI). IEEE, pp 1–4. https://doi.org/10.1109/CBMI.2018.8516556

  7. Kar T, Kanungo P (2017) Video shot boundary detection based on Hilbert and wavelet transform. In: 2017 2nd International Conference on Man and Machine Interfacing (MAMI). IEEE, pp 1–6. https://doi.org/10.1109/MAMI.2017.8307865

  8. Kar T, Kanungo P (2022) A gradient based dual detection model for shot boundary detection. Multimed Tools Appl https://doi.org/10.1007/s11042-022-13547-y

  9. LPG G, Dominic S (2014) Walsh–Hadamard transform kernel-based feature vector for shot boundary detection. IEEE Trans Image Process 23:5187–5197. https://doi.org/10.1109/TIP.2014.2362652

    Article  MathSciNet  Google Scholar 

  10. Mishra R (2021) Video shot boundary detection using hybrid dual tree complex wavelet transform with Walsh Hadamard transform. Multimed Tools Appl 80:28109–28135. https://doi.org/10.1007/s11042-021-11052-2

    Article  Google Scholar 

  11. Mondal J, Kundu MK, Das S, Chowdhury M (2018) Video shot boundary detection using multiscale geometric analysis of nsct and least squares support vector machine. Multimed Tools Appl 77:8139–8161. https://doi.org/10.1007/s11042-017-4707-9

    Article  Google Scholar 

  12. Mussel Cirne MV, Pedrini H (2018) VISCOM: a robust video summarization approach using color co-occurrence matrices. Multimed Tools Appl 77:857–875. https://doi.org/10.1007/s11042-016-4300-7

    Article  Google Scholar 

  13. Nishani E, Cico B (2017) Computer vision approaches based on deep learning and neural networks: deep neural networks for video analysis of human pose estimation. In: 2017 6th Mediterranean conference on embedded computing (MECO). IEEE, pp 1–4

  14. Pal G, Acharjee S, Rudrapaul D, Ashour AS, Dey N (2015) Video segmentation using minimum ratio similarity measurement. Int J Image Min 1:87. https://doi.org/10.1504/IJIM.2015.070027

    Article  Google Scholar 

  15. Rashmi BS, Nagendraswamy HS (2021) Video shot boundary detection using block based cumulative approach. Multimed Tools Appl 80:641–664. https://doi.org/10.1007/s11042-020-09697-6

    Article  Google Scholar 

  16. Roomi SMM, Prakash VJ, Karthikeyan S, Shankar KR (1999) A contrast enhancement technique based on visual significance. J Indian Inst Sci 79(2):89

  17. Sasithradevi A, Mohamed Mansoor Roomi S (2020) A new pyramidal opponent color-shape model based video shot boundary detection. J Vis Commun Image Represent 67:102754. https://doi.org/10.1016/j.jvcir.2020.102754

    Article  Google Scholar 

  18. Shou MZ, Lei SW, Wang W, Ghadiyaram D, Feiszli M (2021) Generic event boundary detection: a benchmark for event segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 8075-8084

  19. Souček T, Lokoč J (2020) Transnet V2: an effective deep network architecture for fast shot transition detection. arXiv preprint arXiv:2008.04838

  20. Souček T, Moravec J, Lokoč J (2019) TransNet: a deep network for fast detection of common shot transitions

  21. Sousa e Santos AC, Pedrini H (2017) Shot boundary detection for video temporal segmentation based on the weber local descriptor. In: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, pp 1310–1315

  22. SRRR AS (2016) Non-subsampled Contourlet transform based shot boundary detection. IJCTA 9:3231–3228

    Google Scholar 

  23. Tang S, Feng L, Kuang Z et al (2019) Fast video shot transition localization with deep structured models. Pp 577–592

  24. Thounaojam DM, Khelchandra T, Singh KM, Roy S (2016) A genetic algorithm and fuzzy logic approach for video shot boundary detection. Comput Intell Neurosci 2016:1–11. https://doi.org/10.1155/2016/8469428

    Article  Google Scholar 

  25. Tippaya S, Sitjongsataporn S, Tan T, Khan MM, Chamnongthai K (2017) Multi-modal visual features-based video shot boundary detection. IEEE Access 5:12563–12575. https://doi.org/10.1109/ACCESS.2017.2717998

    Article  Google Scholar 

  26. Tong W, Song L, Yang X et al (2015) CNN-based shot boundary detection and video annotation. In: 2015 IEEE international symposium on broadband multimedia systems and broadcasting. IEEE, pp 1–5

  27. Wu L, Zhang S, Jian M, Lu Z, Wang D (2019) Two stage shot boundary detection via feature fusion and spatial-temporal convolutional neural networks. IEEE Access 7:77268–77276. https://doi.org/10.1109/ACCESS.2019.2922038

    Article  Google Scholar 

  28. Xu J, Song L, Xie R (2016) Shot boundary detection using convolutional neural networks. In: 2016 Visual Communications and Image Processing (VCIP), Chengdu, China, pp 1–4. https://doi.org/10.1109/VCIP.2016.7805554

  29. Zhang L, Suganthan PN (2016) A comprehensive evaluation of random vector functional link networks. Inf Sci (N Y) 367–368:1094–1105. https://doi.org/10.1016/j.ins.2015.09.025

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre for Advanced Data Science, Vellore Institute of Technology, Chennai, India

    Sasithradevi A

  2. Department of Electronics and Communication Engineering, Thiagarajar College of Engineering, Madurai, India

    S. Mohamed Mansoor Roomi

  3. School of Electronics Engineering, Vellore Institute of Technology, Chennai, India

    P. Nirmala

Authors
  1. Sasithradevi A
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Mohamed Mansoor Roomi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. P. Nirmala
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sasithradevi A.

Ethics declarations

Conflict of interest

The authors declare that they have no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

A, S., Roomi, S.M.M. & Nirmala, P. Visual significance model based temporal signature for video shot boundary detection. Multimed Tools Appl 82, 23037–23054 (2023). https://doi.org/10.1007/s11042-023-14882-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14882-4

Keywords

Search

Navigation