Real Time Hand Tracking and Gesture Recognition as a 3D Input Device for Graphical Applications

Abstract

This paper outlines a system design and implementation of a 3D input device for graphical applications which uses real time hand tracking and gesture recognition to provide the user with an intuitive interface for tomorrow’s applications. Point Distribution Models (PDMs) have been shown to be successful at tracking deformable objects. This system demonstrates how these ‘smart snakes’ can be used in real time with a real world problem. The system is based upon Open Inventor¹ and designed for use with Silicon Graphics Indy Workstations, but provisions have been make for the move to other platforms and applications. We demonstrate how PDMs provide the ideal feature vector for model classification. It is shown how computer vision can provide a low cost, intuitive interface that has few hardware constraints. We also give the reader an insight into the next generation of HCI and Multimedia, providing a 3D scene viewer and VRML browser based upon the handtracker. Further allowances have been made to facilitate the inclusion of the handtracker within third party Inventor applications. All source code, libraries and applications can be downloaded for free from the above web addresses. This paper demonstrates how computer vision and computer graphics can work together providing an interdisciplinary approach to problem solving.