Engin Mendi
ABSTRACT
In this paper, we present a novel algorithm for shot boundary detection and key frame extraction from video sequences. Saliency maps representing the attended regions are produced from the color and luminance features of the video frames. Introducing a novel signal fidelity measurement -saliency based structural similarity index- the similarity of the maps is measured. Based on the similarities, shot boundaries and key frames are determined. Proposed algorithm is tested on neurosurgical videos and precision and recall performances are measured. Experimental results validate effectiveness of the proposed shot boundary detection and key frame extraction algorithm. Moreover, the algorithm is robust to dissolving digital video effects used in shot transition.
- Mendi, E., Bayrak, B. and Yasargil G., "Shot Boundary Detection and Key Frame Extraction from Neurosurgical Video Sequences based on Color Information", The Seventh Annual Conference of the MidSouth Computational Biology and Bioinformatics Society (MCBIOS), Jonesboro, Arkansas, February 19--20, 2010. (submitted).Google Scholar
- Porter, S., "Video Segmentation and Indexing using Motion Estimation". PhD thesis, Department of Computer Science, University of Bristol, 2004.Google Scholar
- Yeo, B. and Liu, B., "Rapid Scene Analysis on Compressed Video". IEEE Transactions on Circuits and Systems for Video Technology, Vol: 5, No: 6, pp: 533--544, December 1995. Google ScholarDigital Library
- Huan, Z., Xiuhuan, L. and Lilei, Y., "Shot Boundary Detection Based on Mutual Information and Canny Edge Detector", Proceedings of the 2008 International Conference on Computer Science and Software Engineering - Volume 02. pp: 1124--1128, July 2008, Las Vegas, Nevada, USA. Google ScholarDigital Library
- Miene, A., Hermes, T., Ioannidis, G., Fathi, R. and Herzog, O., "Automatic shot boundary detection and classification of indoor and outdoor scenes". TREC 2002.Google Scholar
- Mas, J. and Fernandez, G., "Video Shot Boundary Detection Based on Color Histogram", TRECVID Workshop 2003, 2003.Google Scholar
- Cooper, M., Foote, J., Adcock, J. and Casi, S., "Shot boundary detection via similarity analysis". In Proceedings of the TRECVID Workshop. 2003.Google Scholar
- Lowe, D. G., "Three-dimensional object recognition from single two dimensional images". Artificial Intelligence, 31:355--395, 1987. Google ScholarDigital Library
- Gong, Y. and Liu, L., "Video summarization using singular value decomposition," IEEE International Conference on Computer Vision and Pattern Recognition, 2000.Google Scholar
- Yueting, Z., Yong, R., Huang, T. S. and Mehrotra, S., "Adaptive key frame extraction using supervised clustering," IEEE International Conference on Image Processing, 1998.Google Scholar
- Achanta, R., Hemami, S., Estrada, F. and Süsstrunk, S., "Frequency-tuned Salient Region Detection", IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2009.Google ScholarCross Ref
- Sheikh, H. R., Sabir, M. and Bovik, A. C., "A statistical evaluation of recent full reference image quality assessment algorithms," IEEE Trans. Image Process., vol. 15, no. 11, pp. 3440--3451, Nov. 2006. Google ScholarDigital Library
- Mendi, E., Milanova, M., Zhou, Y. and Talburt, J., "Image Quality Assessment based on Salient Region Detection", EURASIP Journal on Advances in Signal Processing, Special Issue on Advanced Image Processing for Defense and Security Applications, 2010. (to appear)Google Scholar
- Li, Q. and Wang, Z., "Video quality assessment by incorporating a motion perception model," in Image Processing, 2007. ICIP 2007. IEEE International Conference on, 2007, vol. 2, pp. 173--176.Google Scholar
- ORLive, Inc.: Online Surgical and Healthcare Video and Webcasts, http://www.orlive.com/.Google Scholar
- NIST, Shot Boundary Evaluation Guide, http://www-nlpir.nist.gov/projects/t2002v/sbmeasures.html.Google Scholar
Index Terms
- Shot boundary detection and key frame extraction using salient region detection and structural similarity
Recommendations
Exploiting contrast cues for salient region detection
Visual saliency detection is an important cue used in human visual system, which can offer efficient solutions for both biological and artificial vision systems. Although there are many saliency detection models that can achieve good results on public ...
Image quality assessment metrics combining structural similarity and image fidelity with visual attention
Image quality assessment has a great importance in several image processing applications. Recently, various objective image quality metrics have been proposed in order to predict human visual perception. In this paper, novel image quality metrics, S-...
Key frame extraction based on visual attention model
Key frame extraction is an important technique in video summarization, browsing, searching and understanding. In this paper, we propose a novel approach to extract the most attractive key frames by using a saliency-based visual attention model that ...
Comments