Gastroscopic image detection is mainly used to diagnose common diseases. Relying on clinicians’ manual identification methods is time-consuming and prone to missing and misdiagnosis. Therefore, for gastroscopic image detection (GID), we propose a novel identification method (GidCNN-SVM) that fuses convolutional neural networks (CNN) with support vector machines (SVM).Firstly, a convolutional neural network (CNN) is used for feature extraction of gastroscopy images to obtain deeper semantic features of gastroscopy images; Secondly, the SVM classifier is used to build the recognition model, so that the SVM based on structural risk minimization will improve the generalization ability and recognition accuracy of the model. Finally, the application of actual gastroscopy images shows that the recognition accuracy of GidCNN-SVM proposed in this paper is as high as 98.2% and the overall AUC value is 98.4%. In gastric cancer and chronic atrophic gastritis, the sensitivity is 93.3% and 95.0%, and the F1 is 98.4% and 98.2%, which is better than that of GoogleNet, AlexNet, LeNet and their methods fused with SVM, and has a wide range of application.