In the field of image classification, traditional feature extraction algorithms, such as texture feature, local feature and global feature, more or less lose some important image classification information, leading to the reduction of the classification effect. Deep learning based on feature pyramid can identify objects of different scales, but it will greatly increase the density of computation and storage. Therefore, we propose a convolutional neural network image classification method based on multi-granularity features. The convolutional neural network model consists of three different channels, each channel uses different granularity convolution kernels to extract multi-granularity feature information, and then uses feature fusion technology for processing. Finally, three granularities of feature information are introduced into the weight parameters to improve the model, and experimental comparisons are made with a variety of single-channel CNN models in CIFAR10 dataset image classification. The experimental results show that the classification accuracy of the model is significantly improved.