M. Zaheer Aziz
ABSTRACT
Standardized methods and metrics for evaluating progress of research is important in every field of science. Computational modeling of visual attention is an important area of research that aims towards machine vision according to the role model of nature. Standards for quantitative evaluation of research achievements in this field are still missing. This paper proposes some measurement methods and metrics that can be used as conventions for evaluation of artificial attention models. The proposed methodology also takes into account the needs of assessing attention under different visual behaviors and considers performance against increasing levels of visual complexity. The measurement methods for the quantities used in the evaluation metrics are designed to make autonomous machine-based evaluation feasible. Creating traces of performance by different attention models using the proposed metrics can provide an objective analysis of the state of the art in this field.
- T. Avraham and M. Lindenbaum. Esaliency (extended saliency): Meaningful attention using stochastic image modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(4):693--708, 2010. Google Scholar
Digital Library
- M. Z. Aziz and B. Mertsching. An attentional approach for perceptual grouping of spatially distributed patterns. In DAGM 2007, LNCS 4713, pages 345--354, Heidelberg - Germany, 2007. Springer. Google ScholarDigital Library
- M. Z. Aziz and B. Mertsching. Fast and robust generation of feature maps for region-based visual attention. Transactions on Image Processing, 17:633--644, 2008. Google ScholarDigital Library
- M. Z. Aziz and B. Mertsching. Towards standardized metrics and methods for evaluation of visual attention models. In WAPCV 2008, pages 180--193, Santorini - Greece, 2008.Google Scholar
- M. Z. Aziz and B. Mertsching. Towards Standardization of Evaluation Metrics and Methods for Visual Attention Models. Attention in Cognitive Systems, LNAI, 5395/2009:227--241, March 2009. Google ScholarDigital Library
- M. Clauss, P. Bayerl, and H. Neumann. A statistical measure for evaluating regions-of-interest based attention algorithms. In DAGM 2004, LNCS 3175, pages 383--390. Springer, 2004.Google ScholarCross Ref
- B. A. Draper and A. Lionelle. Evaluation of selective attention under similarity transforms. In WAPCV 03, 2003.Google Scholar
- S. Frintrop, G. Backer, and E. Rome. Goal-directed search with a top-down modulated computational attention system. In DAGM 2005, LNCS 3663, pages 117--124. Springer, 2005. Google ScholarDigital Library
- N. Hawes and J. Wyatt. Towards context-sensitive visual attention. In Second International Cognitive Vision Workshop (ICVW06), 2006.Google Scholar
- M. Heinen and P. Engel. Evaluation of visual attention models under 2d similarity transformations. In Proceedings of the 2009 ACM symposium on Applied Computing, pages 1156--1160. ACM, 2009. Google ScholarDigital Library
- H. Hügli, T. Jost, and N. Ouerhani. Model performance for visual attention in real 3D color scenes. In IWINAC 2005, pages 469--478. LNCS 3562, Springer, 2005. Google ScholarDigital Library
- L. Itti, U. Koch, and E. Niebur. A model of saliency-based visual attention for rapid scene analysis. Transactions on Pattern Analysis and Machine Intelligence, 20:1254--1259, 1998. Google ScholarDigital Library
- G. Marmitt and A. T. Duchowski. Modeling visual attention in vr:measuring the accuracy of predicted scanpaths. In EUROGRAPHICS 2002, 2002.Google Scholar
- O. L. Meur, P. L. Callet, D. Barba, and D. Thoreau. A coherent computational approach to model bottom-up visual attention. Transactions on Pattern Analysis and Machine Intelligence, 28:802--817, 2006. Google ScholarDigital Library
- T. Michalke, A. Gepperth, M. Schneider, J. Fritsch, and C. Goerick. Towards a human-like vision system for resource-constrained intelligent cars. In ICVS 2007, pages 264--275. Bielefeld University eCollections, Germany, 2004.Google Scholar
- V. Navalpakkam and L. Itti. Modeling the influence of task on attention. Vision Research, pages 205--231, 2005.Google ScholarCross Ref
- G. Osterberg. Topography of the layer of rods and cones in the. human retina. Acta Ophthalmologica, 6:11--102, 1935.Google Scholar
- N. Ouerhani, R. von Wartburg, H. Hügli, and R. Müri. Empirical validation of the saliency-based model of visual attention. Electronic Letters on Computer Vision and Image Analysis, 3(1):13--24, 2004.Google ScholarCross Ref
- R. J. Peters, A. Iyer, L. Itti, and C. Koch. Components of bottom-up gaze allocation in natural images. Vision Research, 45:2397--2416, 2005.Google ScholarCross Ref
- S. Polyak. The Retina. University of Chicago Press, Chicago, USA, 1941.Google Scholar
- C. M. Privitera and L. W. Stark. Algorithms for defining visual regions-of-interest: Comparison with eye fixations. Transactions on Pattern Analysis and Machine Intelligence, 9:970--982, 2000. Google ScholarDigital Library
- M. Riesenhuber. Object recognition in cortex: Neural mechanisms, and possible roles for attention. In Neurobiology of attention, pages 279--287. Elsevier, 2005.Google ScholarCross Ref
- A. L. Yarbus. Eye Movements and Vision. Plenum Press, New York, 1967.Google ScholarCross Ref
- Towards standardization of metrics for evaluation of artificial visual attention
Recommendations
Towards Standardization of Evaluation Metrics and Methods for Visual Attention Models
Attention in Cognitive SystemsEvery field of science requires standardization of metrics and measurement methods for detecting true advancement in research. Efforts on computational models of visual attention models have increased in the recent years and now it is important to have ...
An Axiomatic Analysis of Diversity Evaluation Metrics: Introducing the Rank-Biased Utility Metric
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information RetrievalMany evaluation metrics have been defined to evaluate the effectiveness ad-hoc retrieval and search result diversification systems. However, it is often unclear which evaluation metric should be used to analyze the performance of retrieval systems given ...
Towards Designing Better Session Search Evaluation Metrics
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information RetrievalUser satisfaction has been paid much attention to in recent Web search evaluation studies and regarded as the ground truth for designing better evaluation metrics. However, most existing studies are focused on the relationship between satisfaction and ...
Comments