We propose a binocular motion tracking system by reconstructing the 3D shape of moving objects. This system has a pair of active cameras mounted on the robot arm. The 3D shape is an important feature to track a moving object. To track the moving object, the cameras fixate a point on the object and reconstruct the 3D shape of the object around the fixation point of the two images with the principle of spatio-temporal differentiation analysis. The 3D shape is fed back for the control of the active cameras and the robot arm for motion tracking. The system realizes an accurate and stable tracking of the moving object.