The combination of Raman spectroscopy and deep learning technology provides an automatic, rapid, and accurate scheme for the clinical diagnosis of pathogenic bacteria. However, the accuracy of existing deep learning methods is still limited because of the single and fixed scales of deep neural networks. We propose a deep neural network that can learn multi-scale features of Raman spectra by using the automatic combination of multi-receptive fields of convolutional layers. This model is based on the expert knowledge that the discrimination information of Raman spectra is composed of multi-scale spectral peaks. We enhance the interpretability of the model by visualizing the activated wavenumbers of the bacterial spectrum that can be used for reference in related work. Compared with existing state-of-the-art methods, the proposed method achieves higher accuracy and efficiency for bacterial identification on isolate-level, empiric-treatment-level, and antibiotic-resistance-level tasks. The clinical bacterial identification task requires significantly fewer patient samples to achieve similar accuracy. Therefore, this method has tremendous potential for the identification of clinical pathogenic bacteria, antibiotic susceptibility testing, and prescription guidance.