This paper presents a control mechanism based on the actual distance between the Unmanned Aerial Vehicles (UAV) and the tracking structure for UAV visual tracking of linear ground structures in a low altitude. This control mechanism is called position control briefly in the following sections. The whole technique methods is divided into two parts: structure identification part and navigation control part. The linear structure is considered as a line in camera image, so the Canny Edge Detector (CED) and Probabilistic Hough Transformation (PHT) are used in image processing to recognize edges, identify the line and extract parameters (angle and distance in image). For autonomous tracking, the navigation and control play significant roles which impacts the response time, flight stability and tracing accuracy severely. By position control, the actual distance between the UAV body frame and the detected structure can be calculated. Then the UAV position will be regulated by the variant PID controller to do corrections until the UAV is flying exactly over the linear structure tracked.