Usage of multisensory information in scientific data sensualization

Abstract.

Multisensory scientific data sensualization methods that utilize virtual reality technology permit the use of several human sensations, such as visual, acoustic, and tactile sensation to display numerical data. The purposes of multisensory data sensualization can be classified as follows: (a) representing the relationships between different kinds of data; (b) displaying data utilizing sensory integration; and (c) representing conditions using the compound image. By using multisensory information, computers increase the ability to express data. However, these methods lead us to the question of which sensation should be used to display data most effectively. In this study, a multisensory data sensualization environment was developed in which color, loudness, sound frequency, and air flow pressure could be used to display scientific data. In particular, a wind sensation display prototype using air flow pressure was developed to generate tactile sensation. A basic experiment was conducted on sensory interference when subjects used two kinds of sensations simultaneously. From these results, guidelines for the usage of multisensory information for each purpose is proposed.