skip to main content
10.1145/2662996.2663008acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Compressed-Domain Correlates of Fixations in Video

Published: 07 November 2014 Publication History

Abstract

In this paper we present two compressed-domain features that are highly indicative of saliency in natural video. Their potential to predict saliency is demonstrated by comparing their statistics around human fixation points in a number of videos against the control points selected randomly away from fixations. Using these features, we construct a simple and effective saliency estimation method for compressed video, which utilizes only motion vectors, block coding modes and coded residuals from the bitstream, with partial decoding. The proposed algorithm has been extensively tested on two ground truth datasets using several accuracy metrics. The results indicate its superior performance over several state-of-the-art compressed-domain and pixel-domain algorithms for saliency estimation.

References

[1]
G. Agarwal, A. Anbu, and A. Sinha. A fast algorithm to find the region-of-interest in the compressed MPEG domain. In Proc. IEEE ICME'03, volume 2, pages 133--136, 2003.
[2]
M. G. Arvanitidou, A. Glantz, A. Krutz, T. Sikora, M. Mrak, and A. Kondoz. Global motion estimation using variable block sizes and its application to object segmentation. In Proc. IEEE WIAMIS'09, pages 173--176, 2009.
[3]
A. Borji and L. Itti. State-of-the-art in visual attention modeling. IEEE Trans. Pattern Anal. Mach. Intell., 35(1):185--207, 2013.
[4]
A. Borji, D. N. Sihite, and L. Itti. Quantitative analysis of human-model agreement in visual saliency modeling: A comparative study. IEEE Trans. Image Process., 22(1):55--69, 2013.
[5]
B. Efron and R. Tibshirani. An introduction to the bootstrap, volume 57. CRC press, 1993.
[6]
Y. Fang, W. Lin, Z. Chen, C. M. Tsai, and C. W. Lin. A video saliency detection model in compressed domain. IEEE Trans. Circuits Syst. Video Technol., 24(1):27--38, 2014.
[7]
A. Garcia-Diaz, X. R. Fdez-Vidal, X. M. Pardo, and R. Dosil. Saliency from hierarchical adaptation through decorrelation and variance normalization. Image and Vision Computing, 30(1):51--64, 2012.
[8]
C. Guo and L. Zhang. A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression. IEEE Trans. Image Process., 19(1):185--198, 2010.
[9]
H. Hadizadeh, M. J. Enriquez, and I. V. Bajić. Eye-tracking database for a set of standard video sequences. IEEE Trans. Image Process., 21(2):898--903, Feb. 2012.
[10]
S. Han and N. Vasconcelos. Biologically plausible saliency mechanisms improve feedforward object recognition. Vision Research, 50(22):2295--2307, 2010.
[11]
J. Harel, C. Koch, and P. Perona. Graph-based visual saliency. Advances in neural information processing systems, 19:545--552, 2007.
[12]
Y. Hochberg and A. C. Tamhane. Multiple comparison procedures. John Wiley & Sons, Inc., 1987.
[13]
L. Itti. Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Trans. Image Process., 13(10):1304--1318, 2004.
[14]
L. Itti and P. F. Baldi. Bayesian surprise attracts human attention. Advances in Neural Information Processing Systems, 19:547--554, 2006.
[15]
L. Itti, C. Koch, and E. Niebur. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell., 20(11):1254--1259, 1998.
[16]
Q. G. Ji, Z. D. Fang, Z. H. Xie, and Z. M. Lu. Video abstraction based on the visual attention model and online clustering. Signal Processing: Image Commun., 28(3):241--253, 2013.
[17]
S. H. Khatoonabadi, I. V. Baji_c, and Y. Shan. Comparison of visual saliency models for compressed video. In Proc. IEEE ICIP'14, 2014. Accepted for presentation.
[18]
W. Kim, C. Jung, and C. Kim. Spatiotemporal saliency detection and its applications in static and dynamic scenes. IEEE Trans. Circuits Syst. Video Technol., 21(4):446--456, 2011.
[19]
E. Kreyszig. Introductory mathematical statistics: principles and methods. Wiley New York, 1970.
[20]
Z. Liu, H. Yan, L. Shen, Y. Wang, and Z. Zhang. A motion attention model based rate control algorithm for H. 264/AVC. In The 8th IEEE/ACIS International Conference on Computer and Information Science (ICIS'09), pages 568--573, 2009.
[21]
Y. F. Ma and H. J. Zhang. A new perceived motion based shot content representation. In Proc. IEEE ICIP'01, volume 3, pages 426--429, 2001.
[22]
Y. F. Ma and H. J. Zhang. A model of motion attention for video skimming. In Proc. IEEE ICIP'02, volume 1, pages 129--132, 2002.
[23]
V. Mahadevan and N. Vasconcelos. Biologically inspired object tracking using center-surround saliency mechanisms. IEEE Trans. Pattern Anal. Mach. Intell., 35(3):541--554, 2013.
[24]
A. K. Moorthy and A. C. Bovik. Visual importance pooling for image quality assessment. IEEE J. Sel. Topics Signal Process., 3(2):193--201, 2009. {25} K. Muthuswamy and D. Rajan. Salient motion detection in compressed domain. IEEE Signal Process. Lett., 20(10):996--999, Oct. 2013.
[25]
E. Niebur and C. Koch. Computational architectures for attention. chapter 9, pages 163--186. Cambridge, MA: MIT Press, 1998.
[26]
R. J. Peters, A. Iyer, L. Itti, and C. Koch. Components of bottom-up gaze allocation in natural images. Vision Research, 45(18):2397--2416, 2005.
[27]
P. Reinagel and A. M. Zador. Natural scene statistics at the center of gaze. Network: Computation in Neural Systems, 10:1--10, 1999.
[28]
H. J. Seo and P. Milanfar. Static and space-time visual saliency detection by self-resemblance. Journal of Vision, 9(12):15, 2009.
[29]
A. Sinha, G. Agarwal, and A. Anbu. Region-of-interest based compressed domain video transcoding scheme. In Proc. IEEE ICASSP'04, volume 3, pages 161--164, 2004.
[30]
G. Sullivan, J. Ohm, W.-J. Han, and T. Wiegand. Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol., 22(12):1649--1668, 2012.
[31]
J. A. Swets. Signal detection theory and ROC analysis in psychology and diagnostics: Collected papers. Lawrence Erlbaum Associates, Inc., 1996.
[32]
T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra. Overview of the H. 264/AVC video coding standard. IEEE Trans. Circuits Syst. Video Technol., 13(7):560--576, 2003.

Cited By

View all
  • (2017)Compressed-domain visual saliency modelsMultimedia Tools and Applications10.1007/s11042-016-4124-576:24(26297-26328)Online publication date: 1-Dec-2017
  • (2016)On the robustness of action recognition methods in compressed and pixel domain2016 6th European Workshop on Visual Information Processing (EUVIP)10.1109/EUVIP.2016.7764584(1-6)Online publication date: Oct-2016
  • (2016)Compressed domain video saliency detection using global and local spatiotemporal featuresJournal of Visual Communication and Image Representation10.1016/j.jvcir.2015.12.01135:C(169-183)Online publication date: 1-Feb-2016
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
PIVP '14: Proceedings of the 1st International Workshop on Perception Inspired Video Processing
November 2014
52 pages
ISBN:9781450331258
DOI:10.1145/2662996
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 November 2014

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. compressed-domain processing
  2. fixations
  3. visual saliency

Qualifiers

  • Research-article

Funding Sources

Conference

MM '14
Sponsor:
MM '14: 2014 ACM Multimedia Conference
November 7, 2014
Florida, Orlando, USA

Acceptance Rates

PIVP '14 Paper Acceptance Rate 6 of 9 submissions, 67%;
Overall Acceptance Rate 6 of 9 submissions, 67%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2017)Compressed-domain visual saliency modelsMultimedia Tools and Applications10.1007/s11042-016-4124-576:24(26297-26328)Online publication date: 1-Dec-2017
  • (2016)On the robustness of action recognition methods in compressed and pixel domain2016 6th European Workshop on Visual Information Processing (EUVIP)10.1109/EUVIP.2016.7764584(1-6)Online publication date: Oct-2016
  • (2016)Compressed domain video saliency detection using global and local spatiotemporal featuresJournal of Visual Communication and Image Representation10.1016/j.jvcir.2015.12.01135:C(169-183)Online publication date: 1-Feb-2016
  • (2015)A novel video saliency map detection model in compressed domainMILCOM 2015 - 2015 IEEE Military Communications Conference10.1109/MILCOM.2015.7357435(157-162)Online publication date: 26-Oct-2015
  • (2015)Video saliency map detection based on global motion estimation2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)10.1109/ICMEW.2015.7169845(1-6)Online publication date: Jun-2015
  • (2015)Video saliency detection incorporating temporal information in compressed domainImage Communication10.1016/j.image.2015.07.01338:C(32-44)Online publication date: 1-Oct-2015
  • (2015)Compressed-domain correlates of human fixations in dynamic scenesMultimedia Tools and Applications10.1007/s11042-015-2802-374:22(10057-10075)Online publication date: 1-Nov-2015

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media