Ryo Yonetani
ABSTRACT
We present a method to analyze a relationship between eye movements and saliency dynamics in videos for estimating attentive states of users while they watch the videos. The multi-mode saliency-dynamics model (MMSDM) is introduced to segment spatio-temporal patterns of the saliency dynamics into multiple sequences of primitive modes underlying the saliency patterns. The MMSDM enables us to describe the relationship by the local saliency dynamics around gaze points, which is modeled by a set of distances between gaze points and salient regions characterized by the extracted modes. Experimental results show the effectiveness of the proposed model to classify the attentive states of users by learning the statistical difference of the local saliency dynamics on gaze-paths at each level of attentiveness.
- Brandherm, B., Prendinger, H., and Ishizuka, M. 2007. Interest estimation based on dynamic bayesian networks for visual attentive presentation agents. In Proc. of ICMI, 346--349. Google Scholar
Digital Library
- Bregler, C. 1997. Learning and recognizing human dynamics in video sequences. In Proc. of CVPR, 568--574. Google ScholarDigital Library
- Calder, A., Lawrence, A., Keane, J., Scott, S., Owen, A., Christoffels, I., and Young, A. 2002. Reading the mind from eye gaze. Neuropsychologia 40, 8, 1129--1138.Google ScholarCross Ref
- Dorr, M., Martinetz, T., Gegenfurtner, K., and Barth, E. 2010. Variability of eye movements when viewing dynamic natural scenes. Journal of vision 10, 10, 1--17.Google ScholarCross Ref
- Doshi, A., and Trivedi, M. M. 2010. Attention estimation by simultaneous observation of viewer and view. In Proc. of CVPR Workshop, 21--27.Google Scholar
- Fletcher, L., and Zelinsky, A. 2009. Driver inattention detection based on eye gaze-road event correlation. International Journal of Robotics Research 28, 6, 774--801. Google ScholarDigital Library
- Goldstein, R. B., Woods, R. L., and Peli, E. 2007. Where people look when watching movies: do all viewers look at the same place? Computers in Biology and Medicine 37, 7, 957--64. Google ScholarDigital Library
- Hirayama, T., Dodane, J.-B., Kawashima, H., and Matsuyama, T. 2010. Estimates of user interest using timing structures between proactive content-display updates and eye movements. IEICE Trans. on Information and Systems E-93D, 6, 1470--1478.Google Scholar
- Itti, L., Koch, C., and Niebur, E. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. on PAMI 20, 11, 1254--1259. Google ScholarDigital Library
- Ji, Q., Lan, P., and Looney, C. 2006. A probabilistic framework for modeling and real-time monitoring human fatigue. IEEE Trans. on Systems, Man and Cybernetics, Part A: Systems and Humans 36, 5, 862--875. Google ScholarDigital Library
- Just, M., and Carpenter, P. 1976. Eye fixations and cognitive processes. Cognitive Psychology 480, 441--480.Google ScholarCross Ref
- Kahneman, D. 1973. Attention and effort. Prentice Hall.Google Scholar
- Kawashima, H., and Matsuyama, T. 2005. Multiphase learning for an interval-based hybrid dynamical system. IEICE Trans. on Fundamentals E88-A, 11, 3022--3035. Google ScholarDigital Library
- Li, Y., Wang, T., and Shum, H.-Y. 2002. Motion texture: a two-level statistical model for character motion synthesis. ACM Trans. on Graphics 21, 3, 465--472. Google ScholarDigital Library
- Munn, S. M., Stefano, L., and Pelz, J. B. 2008. Fixation-identification in dynamic scenes. In Proc. of APGV, 33--42. Google ScholarDigital Library
- Nakano, Y. I., and Ishii, R. 2010. Estimating user's engagement from eye-gaze behaviors in human-agent conversations. In Proc. of IUI, 139--148. Google ScholarDigital Library
- North, B., Blake, A., Isard, M., and Rittscher, J. 2000. Learning and classification of complex dynamics. IEEE Trans. on PAMI 22, 9, 1016--1034. Google ScholarDigital Library
- Pavlovic, V., Rehg, J. M., and Maccormick, J. 2000. Learning switching linear models of human motion. In Proc. of NIPS, 981--987.Google Scholar
- Qvarfordt, P., and Zhai, S. 2005. Conversing with the user based on eye-gaze patterns. In Proc. of CHI, 221--230. Google ScholarDigital Library
- Schneider, W., and Shiffrin, R. M. 1977. Controlled and automatic human information processing: I. detection, search, and attention. Psychological Review 84, 1--66.Google ScholarCross Ref
- Walther, D., and Koch, C. 2006. Modeling attention to salient proto-objects. Neural Networks: the Official Journal of the International Neural Network Society 19, 9, 1395--1407. Google ScholarDigital Library
- Wildes, R. P. 1998. A measure of motion salience for surveillance applications. In Proc. of ICIP, 183--187.Google ScholarCross Ref
Index Terms
- Multi-mode saliency dynamics model for analyzing gaze and attention
Recommendations
A neural network implementation of a saliency map model
The saliency map model proposed by Itti and Koch [Itti, L., & Koch, C. (2000). A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Research, 40, 1489-1506] has been a popular model for explaining the guidance of ...
Perceptually guided high-fidelity rendering exploiting movement bias in visual attention
A major obstacle for real-time rendering of high-fidelity graphics is computational complexity. A key point to consider in the pursuit of “realism in real time” in computer graphics is that the Human Visual System (HVS) is a fundamental part of the ...
A saliency-based bottom-up visual attention model for dynamic scenes analysis
This work proposes a model of visual bottom-up attention for dynamic scene analysis. Our work adds motion saliency calculations to a neural network model with realistic temporal dynamics [(e.g., building motion salience on top of De Brecht and Saiki ...
Comments