Gestalt-based composition and performance in multimodal environments

Abstract

The paper introduces the concept of multimodal environments in relation to Gestalt theory. Multimedia environments provide digital extensions for different human activities, such as movement, thinking, composing, listening, planning, etc. The environments discussed in this paper make use of the state-of-the art in sensoring technology and computing. They typically combine different methods of knowledge representation such as symbolic, iconic, and subsymbolic representations into a hybrid architecture. Movement detection, as well as beat induction and musical responses to action that happen on the scene all involve Gestalt notions. In this paper, we focus on a number of requirements for the application of multimodal environments in music and art. Applications are discussed and the basic architecture of an existing experimental platform is outlined.