Abstract
Based on the previous outcomes [1], the study utilized a laboratory experiment including 30 participants to explore the impact of color-touch cross-modal correspondence on the temporal integration of multimodal information. This experiment adopted an intra-group design, and all participants were required to complete a series of multimodal perception tasks and stimulation level judgment tasks in the same environment. In each trial, a vibration stimulus, and a color stimulus, each lasting 100 ms would appear. There might be a stimulus onset asynchrony (SOA) between two stimuli. After the stimuli combination, participants had to finish a temporal judgment task to determine the order in which two stimuli appeared. The independent variables of this experiment were stimuli combination type and SOA. Among them, the stimuli combination type is also tied with color chroma level (2 levels, high or low), vibration amplitude level (2 levels, high or low), and cross-modal correspondence (2 levels, high or low). There were two dependent variables in the experiment to evaluate the judgment process, namely accuracy and reaction time. As for results, firstly, cross-modal correspondence had no significant impact on temporal judgment. Besides, other independent variables (SOA, stimuli combinations, vibration amplitude levels, color chroma levels and the sequence of stimuli) had some impacts on accuracy and reaction time (RT). Although the experiment did not discover the relationship between color-touch cross-modal correspondence and temporal integration of multimodal information, further research could adjust the experimental design based on this to further explore related fields.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Yuan, T., Rau, P.-L., Zhao, J., Zheng, J.: Color-touch cross-modal correspondence and its impact on single-modal judgement in multimodal perception. Multisensory Res. 36, 1–25 (2023). https://doi.org/10.1163/22134808-bja10098
Lewkowicz, D., Turkewitz, G.: Cross-modal equivalence in early infancy: auditory-visual intensity matching. Dev. Psychol. 16, 597–607 (1980). https://doi.org/10.1037/0012-1649.16.6.597
Lalanne, C., Lorenceau, J.: Crossmodal integration for perception and action. J. Physiol. Paris 98, 265–279 (2004). https://doi.org/10.1016/j.jphysparis.2004.06.001
Grifoni, P.: Multimodal human computer interaction and pervasive services (2009). https://doi.org/10.4018/978-1-60566-386-9
Spence, C.: Crossmodal correspondences: a tutorial review. Atten. Percept. Psychophys. 73, 971–995 (2011). https://doi.org/10.3758/s13414-010-0073-7
Alais, D., Burr, D.: The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol. CB. 14, 257–262 (2004). https://doi.org/10.1016/j.cub.2004.01.029
Driver, J., Spence, C.: Attention and the crossmodal construction of space. Trends Cogn. Sci. 2, 254–262 (1998). https://doi.org/10.1016/S1364-6613(98)01188-7
Dong, C., Wang, Z., Li, R., Wang, S.: Individual variations in McGurk illusion susceptibility reflect different integration-segregation strategies of audiovisual speech perception (2023). https://doi.org/10.1101/2023.12.15.571270
Gilbert, A., Martin, R., Kemp, S.: Cross-modal correspondence between vision and olfaction: the color of smells. Am. J. Psychol. 109, 335–351 (1996). https://doi.org/10.2307/1423010
Spence, C., Levitan, C., Shankar, M., Zampini, M.: Does food color influence taste and flavor perception in humans? Chemosens. Percept. 3, 68–84 (2010). https://doi.org/10.1007/s12078-010-9067-z
Zampini, M., Sanabria, D., Phillips, N., Spence, C.: The multisensory perception of flavor: assessing the influence of color cues on flavor discrimination responses. Food Qual. Prefer. 18, 975–984 (2007). https://doi.org/10.1016/j.foodqual.2007.04.001
Al-Ayash, A., Kane, R., Smith, D., Green‐Armytage, P.: The influence of color on student emotion, heart rate, and performance in learning environments (2016)
Martino, G., Marks, L.: Cross-modal interaction between vision and touch: the role of synesthetic correspondence. Perception 29, 745–754 (2000). https://doi.org/10.1068/p2984
Ludwig, V., Simner, J.: What color does that feel? Tactile-visual mapping and the development of cross-modality. Cortex J. Devoted Study Nerv. Syst. Behav. 49 (2012). https://doi.org/10.1016/j.cortex.2012.04.004
Elliot, A., Aarts, H.: Perception of the color red enhances the force and velocity of motor output. Emot. Wash. DC. 11, 445–449 (2011). https://doi.org/10.1037/a0022599
Kahol, K., French, J., Bratton, L., Panchanathan, S.: Learning and perceiving colors haptically (2006). https://doi.org/10.1145/1168987.1169017
Delazio, A., Israr, A., Klatzky, R.: Cross-modal correspondence between vibrations and colors (2017). https://doi.org/10.1109/WHC.2017.7989904
Slobodenyuk, N., Jraissati, Y., Kanso, A., Ghanem, L., Elhajj, I.: Cross-modal associations between color and haptics. Atten. Percept. Psychophys. 68 (2015). https://doi.org/10.3758/s13414-015-0837-1
Jaśkowski, P.: Temporal-order judgment and reaction time for short and long stimuli. Psychol. Res. 54, 141–145 (1992). https://doi.org/10.1007/BF00922093
Gibbon, J., Rutschmann, R.: Temporal order judgment and reaction time. Science 165, 413–415 (1969). https://doi.org/10.1126/science.165.3891.413
Lable, I.: Conscious and unconscious processes: psychodynamic, cognitive, and neurophysiological convergences. Neuropsychoanalysis Interdiscip. J. Psychoanal. Neurosci. 2, 99–102 (2014). https://doi.org/10.1080/15294145.2000.10773290
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
The authors have no competing interests to declare that are relevant to the content of this article.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yuan, T., Rau, PL.P. (2024). The Impact of Color-Touch Cross-Modal Correspondence on the Temporal Integration of Multimodal Information. In: Rau, PL.P. (eds) Cross-Cultural Design. HCII 2024. Lecture Notes in Computer Science, vol 14699. Springer, Cham. https://doi.org/10.1007/978-3-031-60898-8_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-60898-8_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-60897-1
Online ISBN: 978-3-031-60898-8
eBook Packages: Computer ScienceComputer Science (R0)