PaSt: Human Tracking and Gestures Recognition for Flexible Virtual Environments Management

Abstract

This paper presents a CAVE-like architecture to support the interaction for small groups of people with a leader in a multi-projection environment in the unusual condition where a vertical depth camera records people and their movements. In this framework, modelling people as gaussians, we localise and track people when they step into a defined area. We compared our approach with a typical local minimum one and our algorithm results to be faster and more accurate. Detected leaders manage the interaction with hands. We developed a trained gesture recognition model and a rule-based one and the former approach reports better outcomes. While the proposed virtual environment is mainly intended as a multi-projection system, the presented architecture allows to dynamically change the area such as to integrate further input and output devices. It can be extended up to provide support in collaborative tasks for remotely connected groups acting in the same virtual room. The whole system has been adopted in Cultural Heritage scenarios to provide an immersive experience for art, historical contents or virtual environments. Interviews with people participating to the experimentation phase of the OrCHeSTRA project show that the system was well-received by the general public and that future extensions towards collaborative environments are encouraged by the end-users.