Minimally invasive surgery (MIS) is a surgical treatment with small scar and quick recovery. In MIS, an endoscope is usually carried by surgeon or assistant to observe surgical instruments and focal points. This paper proposes a new camera tracking system to help surgeons observe the operating environment in MIS. Held by the mirror-holding robot, the camera follows the specified tool and focal points intelligently and autonomously. Based on the target in the picture taken by the camera, the system adjusts the camera position, which is dynamic operation field tracking. It helps surgeons observe the operation in real time. The tracking algorithm can ensure the accuracy, safety and real-time tracking of the motion of the camera. The simulation results verified the effectiveness of the algorithm.