Abstract
The computation of visual attention is an exhaustive procedure to locate conspicuous regions within a frame, which contrast with the surrounding background. In this paper we propose a unique algorithm to estimate visual saliency in the compressed domain using intra-coded frames from High Efficiency Video Coding (HEVC) encoded video sequences. By exclusively combining data obtained from the coding unit structure, intra mode block predictions and the residual data, a visual saliency approximation is obtained. The proposed model can accurately detect salient regions without the need to fully decode the HEVC bitstream. Experimental results show the proposed algorithm compares positively against multiple methods in the literature, highlighting accurate saliency detection with minimal time additions to the video coding computation. The new methodology can provide aid to a wide variety of fields such as advertising, watermarking, video editing and spatial-temporal adaptation.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Cognitive Psychology 12(1), 97–136 (1980)
Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. on Pattern Analysis and Machine Intelligence 20(11), 1254–1259 (1998)
Riche, N., Mancas, M., Gosselin, B., Dutoit, T.: Rare: a new bottom-up saliency model. In: Proceedings of the International Conference on Image Processing (IEEE ICIP 2012), Orlando, USA, pp. 1–4 (2012)
Ngau, C., Ang, L., Seng, K.: Bottom-up visual saliency map using wavelet transform domain. In: 2010 3rd IEEE International Conference on Computer Science and Information Technology (ICCSIT), vol. 1, pp. 692–695 (July 2010)
Kim, J., Yi, C., Kim, T.: Roi-centered compression by adaptive quantization for sports video. IEEE Trans. on Consumer Electronics 56(2) (May 2010)
Wiegand, T., Sullivan, G.J.: The h.264/avc video coding standard. IEEE Signal Processing Magazine, 148–153 (March 2007)
Sullivan, G., Ohm, J., Han, W., Wiegand, T.: Overview of the high efficiency video coding (hevc) standard. IEEE Trans. on Circuits and Systems for Video Technology (99), 1 (2012)
Oakes, M., Bhowmik, D., Abhayaratne, C.: Visual attention-based watermarking. In: 2011 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 2653–2656 (2011)
Oakes, M., Abhayaratne, C.: Visual saliency estimation for video. In: 2012 13th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS), pp. 1–4 (2012)
Li, Z., Fang, T., Huo, H.: A saliency model based on wavelet transform and visual attention. SCIENCE CHINA Information Sciences 53, 738–751 (2010)
Achanta, R., Hemami, S., Estrada, F., Süsstrunk, S.: Frequency-tuned Salient Region Detection. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1597–1604 (2009)
Lainema, J., Bossen, F., Han, W.J., Min, J., Ugur, K.: Intra coding of the hevc standard. IEEE Trans. on Circuits and Systems for Video Technology 22(12), 1792–1801 (2012)
Tan, T.K., Kanumuri, S., Bossen, F.: Enhancements to intra coding jctvc-d235. JCTVC-D235, Daegu, Korea (2011)
Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., Shum, H.Y.: Learning to detect a salient object. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(2), 353–367 (2011)
MacKay, D.J.C.: Information Theory, Inference and Learning Algorithms. Cambridge University Press, New York (2002)
Mitra, S., Sicuranza, G.: Region-based filtering of images and video sequences: A morphological viewpoint. In: Nonlinear Image Processing, pp. 249–288. Academic Press (2001)
Gide, M.S., Karam, L.J.: Comparative evaluation of visual saliency models for quality assessment task. In: 6th International Workshop on Video Processing and Quality Metrics for Consumer Electronics, pp. 37–40 (2012)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Oakes, M., Abhayaratne, C. (2014). A New Saliency Model Using Intra Coded High Efficiency Video Coding (HEVC) Frames. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds) MultiMedia Modeling. MMM 2014. Lecture Notes in Computer Science, vol 8325. Springer, Cham. https://doi.org/10.1007/978-3-319-04114-8_49
Download citation
DOI: https://doi.org/10.1007/978-3-319-04114-8_49
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-04113-1
Online ISBN: 978-3-319-04114-8
eBook Packages: Computer ScienceComputer Science (R0)