Abstract
Provisioning preview video for long duration videos has always been an expectation. This paper proposes an approach for user-defined customizable preview video generation using visual saliency by extracting some of the fundamental features of the human visual system such as color, intensity and motion. This motivation led to investigating this option for Vision Dominant videos viz., sports videos, wild-life videos, sea world videos etc. Our proposal for visual saliency computation follows the PQFT (Phase Spectrum of Quaternion Fourier transform) model for feature extraction where phase information is used to detect motion(activity)within the frame. The proposed methodology comprises three main stages, with the pre-processing stage that operates on the dataset to reduce the overall computational complexity. The intermediate stage, the saliency detection stage involves feature extraction algorithm to compute saliency values for each frame generating saliency maps and curves that helps in classifying the videos as low, medium and high activity videos. In the final stage, customization is provisioned based on the user's choice of percentage or duration reduction of the original duration of the video to generate preview with keyframe extraction based on maximum saliency values. The PQFT Model deployed is independent of prior knowledge and other parameters making it computationally simple. The experimental results indicate that the proposed method is capable of generating stable and good results. This novel method finds application in different fields towards automatic summarization facilitating preview video on video hosting and streaming websites.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Evangelopoulos, G., et al.: Video event detection and summarization using audio, visual and text saliency. In: 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 3553–3556 (2009). https://doi.org/10.1109/ICASSP.2009.4960393
Cong, R., Lei, J., Fu, H., Cheng, M.M., Lin, W., Huang, Q.: Review of visual saliency detection with comprehensive information. IEEE Trans. Circuits Syst. Video Technol. 29(10), 2941–2959 (2018)
Ramya, G., Kulkarni, S.: Visual saliency based video summarization: a case study for preview video generation. In: Mandal, J.K., Bhattacharya, K., Majumdar, I., Mandal, S. (eds.) Information, Photonics and Communication. LNNS, vol. 79, pp. 155–165. Springer, Singapore (2020). https://doi.org/10.1007/978-981-32-9453-0_16
Evangelopoulos, G., Rapantzikos, K., Maragos, P., Avrithis, Y., Potamianos, A.: Audiovisual attention modeling and salient event detection. In: Maragos, P., Potamianos, A., Gros, P. (eds.) Multimodal Processing and Interaction. Multimedia Systems and Applications, vol. 33. Springer, Boston (2008). https://doi.org/10.1007/978-0-387-76316-3_8
Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Cognit. Psychol. 12(1), 97–136 (1980). ISSN 0010-0285. https://doi.org/10.1016/0010-0285(80)900055
Guo, C., Zhang, L.: A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression. IEEE Trans. Image Process. 19(1), 185–198 (2010). https://doi.org/10.1109/TIP.2009.2030969
Jadon, S., Jasim, M.: Unsupervised video summarization framework using keyframe extraction and video skimming. In: 2020 IEEE 5th International Conference on Computing Communication and Automation (ICCCA), pp. 140–145 (2020). https://doi.org/10.1109/ICCCA49541.2020.9250764
Castleman, K.: Digital Image Processing. Prentice-Hall, New York (1996)
Schauerte, B., Stiefelhagen, R.: Quaternion-based spectral saliency detection for eye fixation prediction. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, pp. 116–129. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33709-3_9
Otani, M., Nakashima, Y., Rahtu, E., Heikkila, J.: Rethinking the evaluation of video summaries. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7596–7604 (2019)
Ma, M., Met, S., Hou, J., Wan, S., Wang, Z.: Video summarization via temporal collaborative representation of adjacent frames. In: 2017 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS), pp. 164–169 (2017). https://doi.org/10.1109/ISPACS.2017.8266466
Boukadida, H., Berrani, S.A., Gros, P.: Automatically creating adaptive video summaries using constraint satisfaction programming: Application to sport content. IEEE Trans. Circuits Syst. Video Technol. 27(4), 920–934 (2017)
Cong, R., Lei, J., Fu, H., Cheng, M.-M., Lin, W., Huang, Q.: Review of visual saliency detection with comprehensive information. IEEE Trans. Circuits Syst. Video Technol. 29(10), 2941–2959 (2019). https://doi.org/10.1109/TCSVT.2018.2870832
Dang, C., Radha, H.: RPCA-KFE: key frame extraction for video using robust principal component analysis. IEEE Trans. Image Process. 24(11), 3742–3753 (2015). https://doi.org/10.1109/TIP.2015.2445572
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Madan, S., Chauhan, S.S., Khan, A., Kulkarni, S. (2023). Customized Preview Video Generation Using Visual Saliency: A Case Study with Vision Dominant Videos. In: Gupta, D., Bhurchandi, K., Murala, S., Raman, B., Kumar, S. (eds) Computer Vision and Image Processing. CVIP 2022. Communications in Computer and Information Science, vol 1777. Springer, Cham. https://doi.org/10.1007/978-3-031-31417-9_21
Download citation
DOI: https://doi.org/10.1007/978-3-031-31417-9_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-31416-2
Online ISBN: 978-3-031-31417-9
eBook Packages: Computer ScienceComputer Science (R0)