Medical scene graph contributes to cognitive tasks such as question answering. An automatic medical scene graph generator can annotate medical images with scene graphs conveniently. An end-to-end model is proposed, which contains an automatic medical scene graph generator based on medical images and surgery records. The novel model can extract symptom information of the images from surgery records via name entity recognition then convert it into scene graphs. The content and format of the scene graph are pre-defined according to the features of the images. A medical scene graph (MSG) dataset is also produced. The dataset contains 116 scene graphs where each graph includes 6 to 30 entities. Experiments show that our model performs well on making scene graphs of CT coronary angiography. Significantly, the scene graphs have the type, state, location, and index of arteries. To verify the practicability of the MSG dataset, We construct an intelligent model that can generate key diagnostic results. In addition, our medical scene graph generator can be applied to other medical areas to annotate different kinds of medical images intelligently. And the MSG dataset can directly be used in other cognitive visual tasks.