Abstract
Eye movement event detection is a demanding technique in cognitive behavior analysis and HCI. Since an eye movement trajectory is a natural path, we try to introduce path signature (PS) to better explore eye movement events; PS is a feature that can highly summarize path information. For this, a multi-input network (MINN) combining 1D-CNN and bidirectional long short-term memory (BiLSTM) is constructed to classify gaze samples as fixation, saccade, smooth pursuit or noise. MINN requires two inputs of local features and global features respectively. The local features include the speed and direction of the gaze trajectory and the global features are the PS’s of the gaze trajectory. Experiments on GazeCom, the biggest eye movement event detection dataset, show that our approach with PS outperforms the state-of-the-art methods that do not use PS.
Supported by Science and Technology Planning Projects of Guangdong Province, China, with grant numbers 2019B010150002 and 2020B0101130019.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anantrasirichai, N., Gilchrist, I.D., Bull, D.R.: Fixation identification for low-sample-rate mobile eye trackers. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 3126–3130. IEEE (2016)
Chen, K.T.: Integration of paths-a faithful representation of paths by noncommutative formal power series. Trans. Am. Math. Soc. 89(2), 395–407 (1958)
Chevyrev, I., Kormilitzin, A.: A primer on the signature method in machine learning. arXiv preprint arXiv:1603.03788 (2016)
Dorr, M., Martinetz, T., Gegenfurtner, K.R., Barth, E.: Variability of eye movements when viewing dynamic natural scenes. J. Vis. 10(10), 28 (2010)
Gyurkó, L.G., Lyons, T., Kontkowski, M., Field, J.: Extracting information from the signature of a financial data stream. arXiv preprint arXiv:1307.7244 (2013)
Hoppe, S., Bulling, A.: End-to-end eye movement detection using convolutional neural networks. arXiv preprint arXiv:1609.02452 (2016)
Komogortsev, O.V., Karpov, A.: Automated classification and scoring of smooth pursuit eye movements in the presence of fixations and saccades. Behav. Res. Methods 45(1), 203–215 (2012). https://doi.org/10.3758/s13428-012-0234-9
Lai, S., Jin, L.: Offline writer identification based on the path signature feature. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 1137–1142. IEEE (2019)
Larsson, L., Nyström, M., Andersson, R., Stridh, M.: Detection of fixations and smooth pursuit movements in high-speed eye-tracking data. Biomed. Signal Process. Control 18, 145–152 (2015)
Li, C., Zhang, X., Liao, L., Jin, L., Yang, W.: Skeleton-based gesture recognition using several fully connected layers with path signature features and temporal transformer module. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 8585–8593 (2019)
Lyons, T., Nejad, S., Arribas, I.P.: Nonparametric pricing and hedging of exotic derivatives. arXiv preprint arXiv:1905.00711 (2019)
Lyons, T., Nejad, S., Arribas, I.P.: Numerical method for model-free pricing of exotic derivatives using rough path signatures. arXiv preprint arXiv:1905.01720 (2019)
Lyons, T., Ni, H., Oberhauser, H.: A feature set for streams and an application to high-frequency financial tick data. In: Proceedings of the 2014 International Conference on Big Data Science and Computing, pp. 1–8 (2014)
Lyons, T., Qian, Z., Qian, Z., et al.: System Control and Rough Paths. Oxford University Press, Oxford (2002)
Lyons, T.J., Caruana, M., Lévy, T.: Differential Equations Driven by Rough Paths. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-71285-5
Lyons, T.J., Sidorova, N.: Sound compression-a rough path approach. Signs 10(1), X1 (2005)
Salvucci, D.D., Goldberg, J.H.: Identifying fixations and saccades in eye-tracking protocols. In: Proceedings of the 2000 Symposium on Eye Tracking Research & Applications, pp. 71–78 (2000)
Startsev, M., Agtzidis, I., Dorr, M.: 1D CNN with BLSTM for automated classification of fixations, saccades, and smooth pursuits. Behav. Res. Methods 51(2), 556–572 (2019)
Tafaj, E., Kasneci, G., Rosenstiel, W., Bogdan, M.: Bayesian online clustering of eye movement data. In: Proceedings of the Symposium on Eye Tracking Research and Applications, pp. 285–288 (2012)
Vidal, M., Bulling, A., Gellersen, H.: Detection of smooth pursuits using eye movement shape features. In: Proceedings of the Symposium on Eye Tracking Research and Applications, pp. 177–180 (2012)
Xie, Z., Sun, Z., Jin, L., Feng, Z., Zhang, S.: Fully convolutional recurrent network for handwritten Chinese text recognition. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 4011–4016. IEEE (2016)
Yang, W., Jin, L., Liu, M.: Character-level Chinese writer identification using path signature feature, dropstroke and deep CNN. arXiv preprint arXiv:1505.04922 (2015)
Yang, W., Jin, L., Liu, M.: Deepwriterid: an end-to-end online text-independent writer identification system. IEEE Intell. Syst. 31(2), 45–53 (2016)
Yang, W., Jin, L., Tao, D., Xie, Z., Feng, Z.: Dropsample: a new training method to enhance deep convolutional neural networks for large-scale unconstrained handwritten Chinese character recognition. Pattern Recogn. 58, 190–203 (2016)
Yang, W., Jin, L., Xie, Z., Feng, Z.: Improved deep convolutional neural network for online handwritten Chinese character recognition using domain-specific knowledge. In: 2015 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 551–555. IEEE (2015)
Yang, W., Lyons, T., Ni, H., Schmid, C., Jin, L.: Developing the path signature methodology and its application to landmark-based human action recognition. arXiv preprint arXiv:1707.03993 (2017)
Zemblys, R., Niehorster, D.C., Komogortsev, O., Holmqvist, K.: Using machine learning to detect events in eye-tracking data. Behav. Res. Methods 50(1), 160–181 (2017). https://doi.org/10.3758/s13428-017-0860-3
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Gao, H., Zhan, Y., Ma, F., Chen, Z. (2021). Eye Movement Event Detection Based onPath Signature. In: Peng, Y., Hu, SM., Gabbouj, M., Zhou, K., Elad, M., Xu, K. (eds) Image and Graphics. ICIG 2021. Lecture Notes in Computer Science(), vol 12889. Springer, Cham. https://doi.org/10.1007/978-3-030-87358-5_67
Download citation
DOI: https://doi.org/10.1007/978-3-030-87358-5_67
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87357-8
Online ISBN: 978-3-030-87358-5
eBook Packages: Computer ScienceComputer Science (R0)