Tunnel moving target detection is subject to the influence of light condition and motion blur, the traditional motion detection method based on pixel points can not be very good at segmenting moving target. To solve this problem, frame difference detection method based on local structure of image and gray level information is proposed. The local mean difference information of the image is calculated by the improved algorithm, and then the similarity measure function and the gray scale measure function are constructed. The similarity measure function effectively describes the structural features of moving objects, and reduces the influence of image background information. The gray scale function is better to highlight the contrast of the target brightness, to increase the division of the target area and the background parts, and to realize the moving target detection correctly. The experimental results show that the detection method of the fusion structure and the gray level information can effectively segment the moving object.