Computer vision techniques have been widely applied to immersive and perceptual human-computer interaction for applications like computer gaming, education, and entertainment. In this paper, relevant techniques are surveyed in terms of image capturing, normalization, motion detection, tracking, feature representation and recognition. In addition, applications of vision techniques in HCI in computer gaming are also summarized in several categories including vision enabled pointing and positioning, vision for manipulating objects, training and education, and miscellaneous applications. The characteristics of existing work are analyzed and discussed, along with corresponding challenges and future research directions proposed.