Unlike the traditional star pattern recognition algorithms that extract star features as a vector to be matched in a database, in this article, a spider-web image is constructed and a hierarchical convolution neural network (CNN) model is proposed to recognize this spider-web image. Stars are linked with different color lines based on the angle distance to enhance the discrimination of the spider-web images. A training dataset and testing dataset are constructed based on spider-web images for CNN model training. A hierarchical CNN model with two stages, which are designed to perform first step identification and recognize similar spider-web images, respectively, is proposed. Experiment results show that, compared with other algorithms, the proposed algorithm is more robust toward the position noise, magnitude noise, small numbers of stars in the field of view, and the presence of false stars.