Tim Rolff
Frank Steinicke
ABSTRACT
We present a novel view on gaze event classification by redefining it as a time-to-event problem. In contrast to previous models, which consider the classification as discrete events, our redefinition allows for estimating the remaining time until the next saccade event. Therefore, we provide a feature analysis and an initial solution for compensating the latency of wearable eye-trackers build in today’s head-mounted displays. Similar to previous classifiers, we utilize oculomotor features such as velocity, acceleration, and event durations. In total, we analyze 104 extracted features of three datasets and apply different regression methods. We identify optimal window sizes for each feature and extract the importance of all extracted windows using recursive feature elimination. Afterwards, we evaluate the performance of all regressors using earlier selected features. We show that our selected regressors can predict the time-to-event better than the baseline, indicating the potential usage of time-to-event prediction of saccades.
Supplemental Material
Available for Download
- Ioannis Agtzidis, Mikhail Startsev, and Michael Dorr. 2016. Smooth Pursuit Detection Based on Multiple Observers. In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications (Charleston, South Carolina) (ETRA ’16). Association for Computing Machinery, New York, NY, USA, 303–306. https://doi.org/10.1145/2857491.2857521Google Scholar
Digital Library
- Naomi S Altman. 1992. An introduction to kernel and nearest-neighbor nonparametric regression. The American Statistician 46, 3 (1992), 175–185.Google ScholarCross Ref
- Richard Andersson, Linnea Larsson, Kenneth Holmqvist, Martin Stridh, and Marcus Nyström. 2017. One algorithm to rule them all? An evaluation and discussion of ten eye movement event-detection algorithms. Behavior research methods 49, 2 (2017), 616–637.Google Scholar
- Anastasios N Angelopoulos, Julien NP Martel, Amit PS Kohli, Jorg Conradt, and Gordon Wetzstein. 2020. Event based, near eye gaze tracking beyond 10,000 hz. arXiv preprint arXiv:2004.03577(2020).Google Scholar
- Elena Arabadzhiyska, Okan Tarhan Tursun, Karol Myszkowski, Hans-Peter Seidel, and Piotr Didyk. 2017. Saccade landing position prediction for gaze-contingent rendering. ACM Transactions on Graphics (TOG) 36, 4 (2017), 1–12.Google ScholarDigital Library
- Kate Bull and David J Spiegelhalter. 1997. Tutorial in biostatistics survival analysis in observational studies. Statistics in medicine 16, 9 (1997), 1041–1074.Google Scholar
- Asim H Dar, Adina S Wagner, and Michael Hanke. 2021. REMoDNaV: robust eye-movement classification for dynamic stimulation. Behavior research methods 53, 1 (2021), 399–414.Google Scholar
- Brendan David-John, Candace Peacock, Ting Zhang, T. Scott Murdison, Hrvoje Benko, and Tanya R. Jonker. 2021. Towards Gaze-Based Prediction of the Intent to Interact in Virtual Reality. In ACM Symposium on Eye Tracking Research and Applications (Virtual Event, Germany) (ETRA ’21 Short Papers). Association for Computing Machinery, New York, NY, USA, Article 2, 7 pages. https://doi.org/10.1145/3448018.3458008Google ScholarDigital Library
- Michael Dorr, Thomas Martinetz, Karl R Gegenfurtner, and Erhardt Barth. 2010. Variability of eye movements when viewing dynamic natural scenes. Journal of vision 10, 10 (2010), 28–28.Google ScholarCross Ref
- Harris Drucker. 1997. Improving Regressors Using Boosting Techniques. In Proceedings of the Fourteenth International Conference on Machine Learning(ICML ’97). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 107–115.Google ScholarDigital Library
- Yoav Freund and Robert E Schapire. 1997. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55, 1 (1997), 119–139.Google ScholarDigital Library
- Anjith George and Aurobinda Routray. 2016. A score level fusion method for eye movement biometrics. Pattern Recognition Letters 82 (2016), 207–215.Google ScholarDigital Library
- Erika Graf, Claudia Schmoor, Willi Sauerbrei, and Martin Schumacher. 1999. Assessment and comparison of prognostic classification schemes for survival data. Statistics in medicine 18, 17-18 (1999), 2529–2545.Google Scholar
- Isabelle Guyon, Jason Weston, Stephen Barnhill, and Vladimir Vapnik. 2002. Gene selection for cancer classification using support vector machines. Machine learning 46, 1 (2002), 389–422.Google Scholar
- Frank E Harrell Jr, Kerry L Lee, and Daniel B Mark. 1996. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Statistics in medicine 15, 4 (1996), 361–387.Google Scholar
- Kenneth Holmqvist, Marcus Nyström, Richard Andersson, Richard Dewhurst, Halszka Jarodzka, and Joost Van de Weijer. 2011. Eye tracking: A comprehensive guide to methods and measures. OUP Oxford, Oxford, England.Google Scholar
- Zhiming Hu, Andreas Bulling, Sheng Li, and Guoping Wang. 2021. Fixationnet: Forecasting eye fixations in task-oriented virtual environments. IEEE Transactions on Visualization and Computer Graphics 27, 5(2021), 2681–2690.Google ScholarCross Ref
- Zhiming Hu, Sheng Li, Congyi Zhang, Kangrui Yi, Guoping Wang, and Dinesh Manocha. 2020. DGaze: CNN-based gaze prediction in dynamic scenes. IEEE transactions on visualization and computer graphics 26, 5(2020), 1902–1911.Google Scholar
- Hung Hung and Chin-Tsang Chiang. 2010. Estimation methods for time-dependent AUC models with survival data. Canadian Journal of Statistics 38, 1 (2010), 8–26.Google Scholar
- Md Rezaul Karim, M Ataharul Islam, 2019. Reliability and Survival Analysis. Springer, Singapore.Google Scholar
- Oleg V Komogortsev, Denise V Gobert, Sampath Jayarathna, Sandeep M Gowda, 2010. Standardization of automated analyses of oculomotor fixation and saccadic behaviors. IEEE Transactions on biomedical engineering 57, 11 (2010), 2635–2645.Google ScholarCross Ref
- Oleg V Komogortsev and Alex Karpov. 2013. Automated classification and scoring of smooth pursuit eye movements in the presence of fixations and saccades. Behavior research methods 45, 1 (2013), 203–215.Google Scholar
- Eileen Kowler. 2011. Eye movements: The past 25 years. Vision research 51, 13 (2011), 1457–1483.Google Scholar
- Eike Langbehn, Frank Steinicke, Markus Lappe, Gregory F Welch, and Gerd Bruder. 2018. In the blink of an eye: leveraging blink-induced suppression for imperceptible position and orientation redirection in virtual reality. ACM Transactions on Graphics (TOG) 37, 4 (2018), 1–11.Google ScholarDigital Library
- Richard Li, Eric Whitmire, Michael Stengel, Ben Boudaoud, Jan Kautz, David Luebke, Shwetak Patel, and Kaan Akşit. 2020. Optical gaze tracking with spatially-sparse single-pixel detectors. In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 117–126.Google ScholarCross Ref
- Yin Li, Miao Liu, and James M Rehg. 2018. In the eye of beholder: Joint learning of gaze and actions in first person video. In Proceedings of the European Conference on Computer Vision (ECCV). 619–635.Google ScholarDigital Library
- Radha Nila Meghanathan, Cees van Leeuwen, and Andrey R Nikolaev. 2015. Fixation duration surpasses pupil size as a measure of memory load in free viewing. Frontiers in human neuroscience 8 (2015), 1063.Google Scholar
- Antje Nuthmann. 2017. Fixation durations in scene viewing: Modeling the effects of local image features, oculomotor parameters, and task. Psychonomic bulletin & review 24, 2 (2017), 370–392.Google Scholar
- Antje Nuthmann, Tim J Smith, Ralf Engbert, and John M Henderson. 2010. CRISP: a computational model of fixation durations in scene viewing.Psychological review 117, 2 (2010), 382.Google Scholar
- John Platt 1999. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers 10, 3 (1999), 61–74.Google Scholar
- Herbert Robbins and Sutton Monro. 1951. A stochastic approximation method. The annals of mathematical statistics 22, 3 (1951), 400–407.Google Scholar
- Timothy A Salthouse and Cecil L Ellis. 1980. Determinants of eye-fixation duration. The American journal of psychology 93, 2 (1980), 207–234.Google Scholar
- Dario D. Salvucci and Joseph H. Goldberg. 2000. Identifying Fixations and Saccades in Eye-Tracking Protocols. In Proceedings of the 2000 Symposium on Eye Tracking Research & Applications (Palm Beach Gardens, Florida, USA) (ETRA ’00). Association for Computing Machinery, New York, NY, USA, 71–78. https://doi.org/10.1145/355017.355028Google ScholarDigital Library
- Abraham Savitzky and Marcel JE Golay. 1964. Smoothing and differentiation of data by simplified least squares procedures.Analytical chemistry 36, 8 (1964), 1627–1639.Google Scholar
- Ronald W Schafer. 2011. What is a Savitzky-Golay filter?[lecture notes]. IEEE Signal processing magazine 28, 4 (2011), 111–117.Google ScholarCross Ref
- Jasper Snoek, Hugo Larochelle, and Ryan P Adams. 2012. Practical Bayesian Optimization of Machine Learning Algorithms. In Advances in Neural Information Processing Systems, F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger (Eds.). Vol. 25. Curran Associates, Inc., Red Hook, USA. https://proceedings.neurips.cc/paper/2012/file/05311655a15b75fab86956663e1819cd-Paper.pdfGoogle Scholar
- Dave M Stampe. 1993. Heuristic filtering and reliable calibration methods for video-based pupil-tracking systems. Behavior Research Methods, Instruments, & Computers 25, 2 (1993), 137–142.Google ScholarCross Ref
- Mikhail Startsev, Ioannis Agtzidis, and Michael Dorr. 2019. 1D CNN with BLSTM for automated classification of fixations, saccades, and smooth pursuits. Behavior Research Methods 51, 2 (2019), 556–572.Google ScholarCross Ref
- Jan-Philipp Stauffert, Florian Niebling, and Marc Erich Latoschik. 2018. Effects of Latency Jitter on Simulator Sickness in a Search Task. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 121–127. https://doi.org/10.1109/VR.2018.8446195Google ScholarCross Ref
- Jan-Philipp Stauffert, Florian Niebling, and Marc Erich Latoschik. 2020. Latency and Cybersickness: Impact, Causes and Measures. A Review. Frontiers in Virtual Reality 1 (2020), 31.Google Scholar
- Niklas Stein, Diederick C Niehorster, Tamara Watson, Frank Steinicke, Katharina Rifai, Siegfried Wahl, and Markus Lappe. 2021. A comparison of eye tracking latencies among several commercial head-mounted displays. i-Perception 12, 1 (2021), 2041669520983338.Google Scholar
- Frank Steinicke. 2016. Being really virtual. Springer, Heidelberg, Germany.Google Scholar
- Qi Sun, Anjul Patney, Li-Yi Wei, Omer Shapira, Jingwan Lu, Paul Asente, Suwen Zhu, Morgan McGuire, David Luebke, and Arie Kaufman. 2018. Towards virtual reality infinite walking: dynamic saccadic redirection. ACM Transactions on Graphics (TOG) 37, 4 (2018), 1–13.Google ScholarDigital Library
- Gerhard Tutz, Matthias Schmid, 2016. Modeling discrete time-to-event data. Springer, Heidelberg, Germany.Google Scholar
- Giacomo Veneri, Pietro Piu, Francesca Rosini, Pamela Federighi, Antonio Federico, and Alessandra Rufa. 2011. Automatic eye fixations identification based on analysis of variance and covariance. Pattern Recognition Letters 32, 13 (2011), 1588–1593.Google ScholarDigital Library
- R Calen Walshe and Antje Nuthmann. 2021. A Computational Dual-Process Model of Fixation-Duration Control in Natural Scene Viewing. Computational Brain & Behavior 4, 4 (2021), 463–484.Google ScholarCross Ref
- David R Walton, Rafael Kuffner Dos Anjos, Sebastian Friston, David Swapp, Kaan Akşit, Anthony Steed, and Tobias Ritschel. 2021. Beyond blur: Real-time ventral metamers for foveated rendering. ACM Transactions on Graphics 40, 4 (2021), 1–14.Google ScholarDigital Library
- Christopher Williams and Matthias Seeger. 2001. Using the Nyström Method to Speed Up Kernel Machines. In Advances in Neural Information Processing Systems, T. Leen, T. Dietterich, and V. Tresp (Eds.). Vol. 13. MIT Press, Boston, USA. https://proceedings.neurips.cc/paper/2000/file/19de10adbaa1b2ee13f77f679fa1483a-Paper.pdfGoogle Scholar
- Kazuo Yamaguchi. 1991. Event history analysis. Sage, Newbury Park, California.Google Scholar
- Raimondas Zemblys, Diederick C Niehorster, and Kenneth Holmqvist. 2019. gazeNet: End-to-end eye-movement event detection with deep neural networks. Behavior research methods 51, 2 (2019), 840–864.Google Scholar
Recommendations
A Deep Learning Architecture for Egocentric Time-to-Saccade Prediction using Weibull Mixture-Models and Historic Priors
ETRA '23: Proceedings of the 2023 Symposium on Eye Tracking Research and ApplicationsReal-time detection of saccades is of major interest for many applications in human-computer interaction and mixed reality. However, due to relatively low update rates and high latencies of current commercially available eye trackers, gaze events are ...
Multi-view classification of psychiatric conditions based on saccades
Graphical abstractDisplay Omitted HighlightsUsing saccadic information to classify different psychiatric conditions.Classifiers could be used to reduce the possible diagnoses for a given patient.Simple descriptors are extracted from the ...
Survival neural networks for time-to-event prediction in longitudinal study
AbstractTime-to-event prediction has been an important practical task for longitudinal studies in many fields such as manufacturing, medicine, and healthcare. While most of the conventional survival analysis approaches suffer from the presence of censored ...
Comments