Fast Moving targets always are shifted or smeared outside the scene in different images sequence to make video by Circle Synthetic Aperture Radar (SAR). In this paper, a novel moving target tracking approach with the shadow detection and tracking (SDT) is presented based on Convolution Neural Network. Based on the shadow characteristic of moving target in SAR imagery, CNN tracking classification is employed on potential moving target candidates extracted from a sequence of temporal and spatial sub-aperture SAR images to detect and track the moving targets. By the simulation experiments and performance analysis, the validity of the proposed algorithm can be demonstrated. Real data set processing results are provided to demonstrate the effectiveness of the proposed approach.