Abstract
Face image preference is influenced by many factors and can be detected by analyzing eye movement data. When comparing two face images, our gaze shifts within and between the faces. Eye tracking data can give us insights into the cognitive processes involved in forming a preference. In this paper, a gaze tracking dataset is analyzed using three machine learning algorithms (MLA): AdaBoost, Random Forest, and Mixed Group Ranks (MGR) as well as a newly developed machine learning framework called Multi-Layer Combinatorial Fusion (MCF) to predict a subject’s face image preference. Attributes constructed from the dataset are treated as input scoring systems. MCF involves a series of layers that consist of expansion and reduction processes. The expansion process involves performing exhaustive score and rank combinations, while the reduction process uses performance and diversity to select a subset of systems that will be passed onto the next layer of analysis. Performance and cognitive diversity are used in weighted scoring system combinations and system selection. The results outperform the Mixed Group Ranks algorithm, as well as our previous work using pairwise scoring system combinations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Breiman, L.: Random forests. Mach. Learn. 45, 5–32 (2001)
Chuk, T., Chan, A.B., Shimojo, S., Hsiao, J.H.: Mind reading: discovering individual preferences from eye movements using switching hidden Markov models. In: Papafragou, A., Grodner, D., Mirman, D., Trueswell, J.C. (eds.) Proceedings of the 38th Annual Conference of the Cognitive Science Society, p. 182. Cognitive Science Society, Austin (2016)
Fantz, R.L.: The origin of form perception. Sci. Am. 204, 66–72 (1961)
Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55, 119–139 (1997)
Han, J., Kamber, M., Pei, J.: Data Mining: Concepts and Techniques, 3rd edn. Morgan Kaufmann, San Francisco (2012)
Ho, T.K., et al.: Decision combination in multiple classifier systems. IEEE Trans PAMI 16(1), 66–75 (1994)
Hsu, D.F., Chung, Y.S., Kristal, B.S.: Combinatorial fusion analysis: methods and practice of combining multiple scoring systems. In: Hsu, H.H. (ed.) Advanced Data Mining Technologies in Bioinformatics, pp. 1157–1181. Idea Group Inc., Calgary (2006)
Hsu, D.F., Kristal, B.S., Schweikert, C.: Rank-score characteristics (RSC) function and cognitive diversity. In: Yao, Y., Sun, R., Poggio, T., Liu, J., Zhong, N., Huang, J. (eds.) BI 2010. LNCS (LNAI), vol. 6334, pp. 42–54. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15314-3_5
Hsu, D.F., Taksa, I.: Comparing rank and score combination methods for data fusion in information retrieval. Inf. Retr. 8(3), 449–480 (2005)
Khushaba, R., et al.: Consumer neuroscience: assessing the brain response to marketing stimuli using electroencephalogram (EEG) and eye tracking. Expert Syst. Appl. 40(9), 3803–3812 (2013)
König, P., et al.: Eye movements as a window to cognitive processes. J. Eye Mov. Res. 9(5), 1–16 (2016)
Kustiawan, I., Liu, C.-Y., Hsu, D.F.: Vertical Handoff decision using fuzzification and combinatorial fusion. IEEE Commun. Lett. 21, 2089–2092 (2017)
Li, Y., Hsu, D.F., Chung, S.M.: Combination of multiple feature selection methods for text categorization by using combinatorial fusion analysis and rank-score characteristic. Int. J. Artif. Intell. Tools 22(2), 1350001 (2013)
Liao, H.I., Shimojo, S.: Dynamic preference formation via gaze and memory. In: Sharot, T., Dolan, R. (ed.) Neuroscience of Preference and Choice, pp. 277–292. Elsevier Inc. (2012)
Liao, H.I., Yeh, S.L., Shimojo, S.: Novelty vs. familiarity principles in preference decisions: task-context of past experience matters. Front. Psychol. 2(43), 1–8 (2011)
Lin, K.L., et al.: Feature selection and combination criteria for improving accuracy in protein structure prediction. IEEE Trans. Nanobiosci. 6(2), 186–196 (2007)
Little, A.C., Jones, B.C., DeBruine, L.M.: Facial attractiveness: evolutionary based research. Philos. Trans. R. Soc. B: Biol. Sci. 366(1571), 1638–1659 (2011)
Lyons, D.M., Hsu, D.F.: Combining multiple scoring systems for target tracking using rank score characteristics. Inform. Fus. 10(2), 124–136 (2009)
Mandler, G., Nakamura, Y., Van Zandt, B.J.: Exp. Psychol. Learn. Mem. Cogn. 13, 646–648 (1987)
Melnik, O., Vardi, Y., Zhang, C.H.: Mixed group ranks: preference and confidence in classifier combination. IEEE Trans. Pattern Anal. Mach. Intell. 26(8), 973–981 (2004)
Raney, G.E., Campbell, S.J., Bovee, J.C.: Using eye movements to evaluate the cognitive processes involved in text comprehension. J. Vis. Exp. JoVE 83, 50780 (2014)
Schweikert, C., Shimojo, S., Hsu, D.F.: Detecting preferences based on eye movement using combinatorial fusion. In: Wang, Y., et al. (ed.) Proceedings of the 2016 IEEE 15th International Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC 2016), pp. 336–343 (2016)
Schweikert, C., Brown, S., Tang, Z., Smith, P.R., Hsu, D.F.: Combining multiple ChIP-seq peak detection systems using combinatorial fusion. BMC Genom. 13(Suppl 8), S12 (2012)
Shimojo, S., Simion, C., Changizi, M.: Gaze and preference – orienting behavior as a somatic precursor of preference decision. In: Adams, Jr., R.B., Ambady, N., Nakayama, K., Shimojo, S. (eds.) Social Vision, pp. 151–163. Oxford Univ. Press (2011)
Shimojo, S., Simion, C., Shimojo, E., Scheier, C.: Gaze bias both reflects and influences preference. Nat. Neurosci. 6, 1317–1322 (2003)
Wang, Y., Wang, Y.: Cognitive informatics models of the brain. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 36(2), 203–207 (2006)
Yang, J.M., et al.: Consensus scoring for improving enrichment in virtual screening. J. Chem. Inform. Model. 45, 1134–1146 (2005)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Schweikert, C., Gobin, L., Xie, S., Shimojo, S., Frank Hsu, D. (2018). Preference Prediction Based on Eye Movement Using Multi-layer Combinatorial Fusion. In: Wang, S., et al. Brain Informatics. BI 2018. Lecture Notes in Computer Science(), vol 11309. Springer, Cham. https://doi.org/10.1007/978-3-030-05587-5_27
Download citation
DOI: https://doi.org/10.1007/978-3-030-05587-5_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-05586-8
Online ISBN: 978-3-030-05587-5
eBook Packages: Computer ScienceComputer Science (R0)