Collision risk assessment aims to provide a subjective cognitive comprehension of the risk level in driving scenarios, which is critical for the safety of autonomous driving systems. Pedestrian crossing scenarios contain intricate human-vehicle interactions. Hence, it is important to capture the rich relations between traffic entities and to assess the collision risk promptly to ensure safety. Existing studies focus on modeling the spatial relationships between the ego-vehicle and other vehicles in typical traffic scenarios, while ignoring the complex interactions between pedestrians and the ego-vehicle in critical driving scenarios. To address this issue, we propose a novel approach that involves constructing traffic scene graphs with enhanced vehicle-pedestrian interactions, along with introducing an innovative deep model built upon Transformer and GCN for pedestrian collision risk assessment. Specifically, to facilitate spatio-temporal modeling of traffic scene graph sequence, we propose a novel unified framework that integrates Multi-Relation Graph Convolution Network (MR-GCN) and Temporal Transformer Encoder. In addition, two variants of traffic scene graph datasets termed as Interaction-Enhanced Scene Graph (IESG) and None-Interaction-Enhanced Scene Graph (Non-IESG) are created for the purpose of assessing pedestrian collision risk, utilizing the CAP-DATA and JAAD respectively. Experiments are conducted on our newly created traffic scene graph datasets of pedestrian crossing scenes. The results on the IESG dataset show that our model outperforms the baseline model with higher accuracy (94% vs. 84%), higher AUC (98% vs. 89%), and higher F1-score (93% vs. 84%). IESG and Non-IESG datasets are available at https://github.com/Pedestrian-Crossing-Collision-Risk-Assessment-Datasets.