Abstract
People sense the world by exploiting correlations between their physical actions and the changing sensory input that results from those actions. Interfaces that translate non-human sensor data to signals that are compatible with the human senses can therefore augment our abilities to make sense of the world. This insight has recently sparked an increase in projects that explore sensemaking and the creation of novel human experiences across scientific and artistic disciplines. However, there currently exists no constructive dialogue between artists and scientists that conduct research on this topic. In this position paper, we identify the theory and practice of sensory augmentation as a domain that could benefit from such a dialogue. We argue that artistic and scientific methods can complement each other within research on sensory augmentation and identify six thematic starting points for a dialogue between the arts and sciences. We conducted a case study to explore these conjectures, in which we instigated such a dialogue on a small scale. The case study revealed that the six themes we identified as relevant for a dialogue on sensory augmentation emerge rather spontaneously in such a dialogue and that such an exchange may facilitate progress on questions that are central to the theory and practice of sensory augmentation. Overall, this position paper contributes preliminary evidence for the potential of, and a starting point for, a dialogue between the arts and sciences that advances our understanding of sensory augmentation and the development of applications that involve it.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Gombrich, E.H.: Art and Illusion. Phaidon, London (1977)
Schwartzman, M.: See Yourself Sensing: Redefining Human Perception. Black Dog Publishing, London (2011)
Noë, A.: Action in Perception. MIT Press, Cambridge (2004)
Marr, D.: Vision. W.H. Freeman, San Francisco (1982)
O’Regan, J.K., Noë, A.: A sensorimotor account of vision and visual consciousness. Behav. Brain Sci. 24(5), 883–917 (2001)
Noë, A.: Strange Tools: Art and Human Nature. Hill and Wang, New York (2015)
Manning, E., Massumi, B.: Thought in the Act. University of Minnesota Press, Minneapolis (2014)
Van Dartel, M.F.: Aesthetics in the Wild: Art and Design Practices and Pedagogies after the Situated Turn. Avans, Breda (2016)
Biggs, M., Karlsson, H.: Research in the Arts. Routledge, New York (2012)
Froese, T., McGann, M., Bigge, W., Spiers, A., Seth, A.K.: The enactive torch: a new tool for the science of perception. IEEE Trans. Haptics 5(4), 365–375 (2012)
Stratton, G.M.: Vision without inversion of the retinal image. Psychol. Rev. 4, 463–481 (1897)
Kohler, I.: The Formation and Transformation of the Perceptual World. Psychological Issues, vol. 3. International University Press (1964). Monograph 12
Miyauchi, S., et al.: Adaptation to left-right reversed vision rapidly activates ipsilateral visual cortex in humans. J. Physiol. Paris 98, 207–219 (2004)
Bach-y-Rita, P., Collins, C.C., Saunders, S.A., White, B., Scadden, L.: Vision substitution by tactile image projection. Nature 221, 963–964 (1969)
Kercel, S.W., Bach-y-Rita, P.: Noninvasive coupling of electronically generated data into the human nervous system. In: Akay, M. (ed.) Wiley Encyclopedia of Biomedical Engineering, pp. 1960–1974. Wiley (2006)
Kaspar, K., König, S., Schwandt, J., König, P.: The experience of new sensorimotor contingencies by sensory augmentation. Conscious. Cogn. 28, 47–63 (2014)
Ward, J., Meijer, P.: Visual experiences in the blind induced by an auditory sensory substitution device. Conscious. Cogn. 19, 492–500 (2010)
Deroy, O., Auvray, M.: Reading the world through the skin and ears: a new perspective on sensory substitution. Front. Psychol. 3, 457 (2012)
Sha, X.W.: Poiesis and Enchantment in Topological Matter. MIT Press, Cambridge (2013)
Novich, N.: Sound-to-touch sensory substitution and beyond. Ph.D. thesis. Rice University (2015)
De Cupere, P.: Blind smell stick (2012). http://www.blindsmellstick.com/index.php/about. Accessed 2 Nov 2016
De Rooij, A., Broekens, J., Lamers, M.H.: Abstract expressions of affect. Int. J. Synth. Emot. 4(1), 1–31 (2013)
Evers, F.: The Academy of the Senses: Synesthetics in Science, Art and Education. ArtScience Interfaculty Press, The Hague (2012)
Kandinsky, W.: Composition VIII (1924). https://www.guggenheim.org/artwork/1924. Accessed 2 Nov 2016
Ramachandran, V.S., Hubbard, E.M.: Synaesthesia: a window into perception, thought and language. J. Conscious. Stud. 8(12), 3–34 (2001)
Maidenbaum, S.: Practical sensory substitution in real and virtual worlds: development, accessibility and neuroscience. In: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, pp. 211–214 (2015)
Van Dartel, M., Misker, J., Nigten, A., Van der Ster, J.: Virtual reality and augmented reality art explained in terms of sensory-motor coordination. In: Proceedings of 4th International Conference on Enactive Interfaces (2007)
Maidenbaum, S., Chebat, D.R., Levy-Tzedek, S., Namer-Furstenberg, R., Amedi, A.: The effect of expanded sensory range via the EyeCane sensory substitution device on the characteristics of visionless virtual navigation. Multi Sens. Res. 27(5–6), 379–397 (2014)
Froese, T., Spiers, A.: Toward a Phenomenological Pragmatics of Enactive Perception. Cognitive Science Research Papers, 593 (2004)
O’Rourke, K.: Walking and Mapping. MIT Press, Cambridge (2013)
Studio Eyal Burstein: Eye Candy Can Ltd. (2007). http://www.eyalburstein.com/eye-candy/1n5r92f24hltmysrkuxtiuepd6nokx. Accessed 2 Nov 2016
Wallas, G.: The Art of Thought. Harcourt Brace, New York (1926)
Grand Illusions. Reversing goggles. http://www.grand-illusions.com/acatalog/Reversing_Goggles.html. Accessed 19 Apr 2017
de Rooij, A., Jones, S.: Mood and creativity: an appraisal tendency perspective. In: Proceedings of the 9th ACM Conference on Creativity and Cognition, pp. 362–365 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
de Rooij, A. et al. (2018). Sensory Augmentation: Toward a Dialogue Between the Arts and Sciences. In: Brooks, A., Brooks, E., Vidakis, N. (eds) Interactivity, Game Creation, Design, Learning, and Innovation. ArtsIT DLI 2017 2017. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 229. Springer, Cham. https://doi.org/10.1007/978-3-319-76908-0_21
Download citation
DOI: https://doi.org/10.1007/978-3-319-76908-0_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-76907-3
Online ISBN: 978-3-319-76908-0
eBook Packages: Computer ScienceComputer Science (R0)