At present, the application of Electroencephalogram (EEG) signal classification to human intention-behavior prediction has become a hot topic in the brain computer interface (BCI) research field. In recent studies, the introduction of convolutional neural networks (CNN) has contributed to substantial improvements in the EEG signal classification performance. However, there is still a key challenge with the existing CNN-based EEG signal classification methods, the accuracy of them is not very satisfying. This is because most of the existing methods only utilize the feature maps in the last layer of CNN for EEG signal classification, which might miss some local and detailed information for accurate classification. To address this challenge, this paper proposes a multi-scale CNN model-based EEG signal classification method. In this method, first, the EEG signals are preprocessed and converted to time-frequency images using the short-time Fourier Transform (STFT) technique. Then, a multi-scale CNN model is designed for EEG signal classification, which takes the converted time-frequency image as the input. Especially, in the designed multi-scale CNN model, both the local and global information is taken into consideration. The performance of the proposed method is verified on the benchmark data set 2b used in the BCI contest IV. The experimental results show that the average accuracy of the proposed method is 73.9 percent, which improves the classification accuracy of 10.4, 5.5, 16.2 percent compared with the traditional methods including artificial neural network, support vector machine, and stacked auto-encoder.