We developed a system for automatic 3-D human body motion in video surveillance sequences. As the first step, the human body that will be used for the analysis is extracted and tracked based on video object (VO) segmentation technique. Once human body is extracted, this 2-D object, which consists of the merging of several regions, is then analyzed for activity and body part interpretation. The specific 2-D human body shape along with the information derived from maximum probable explanation (MPE) of dynamic Bayesian modeling is then effectively utilized to obtain 3-D body model frame-by-frame so that the 3-D motion trajectory of the human object can be derived for any future event analysis, which is independent of the viewing perspective of the 2-D video