Model-Based Image Segmentation for Multi-view Human Gesture Analysis

Abstract

Multi-camera networks bring in potentials for a variety of vision-based applications through provisioning of rich visual information. In this paper a method of image segmentation for human gesture analysis in multi-camera networks is presented. Aiming to employ manifold sources of visual information provided by the network, an opportunistic fusion framework is described and incorporated in the proposed method for gesture analysis. A 3D human body model is employed as the converging point of spatiotemporal and feature fusion. It maintains both geometric parameters of the human posture and the adaptively learned appearance attributes, all of which are updated from the three dimensions of space, time and features of the opportunistic fusion. In sufficient confidence levels parameters of the 3D human body model are again used as feedback to aid subsequent vision analysis. The 3D human body model also serves as an intermediate level for gesture interpretation in different applications.

The image segmentation method described in this paper is part of the gesture analysis problem. It aims to reduce raw visual data in a single camera to concise descriptions for more efficient communication between cameras. Color distribution registered in the model is used to initialize segmentation. Perceptually Organized Expectation Maximization (POEM) is then applied to refine color segments with observations from a single camera. Finally ellipse fitting is used to parameterize segments. Experimental results for segmentation are illustrated. Some examples for skeleton fitting based on the elliptical segments will also be shown to demonstrate motivation and capability of the model-based segmentation approach for multi-view human gesture analysis.