The Resource-Constrained Project Schedule Problem (RCPSP) is one of the most studied Cumulative Scheduling Problems with many real-world applications. Priority rules are widely adopted in practical RCPSP solving, however traditional rules are manually designed by human experts and may perform poorly. Lately, Deep Reinforcement Learning (DRL) has been shown to be effective in learning dispatching rules for disjunctive scheduling problems. However, research on cumulative problems such as RCPSP is rather sparse. In this paper, we propose an end-to-end DRL method to train high-quality priority rules for RCPSP. Based on its graph structure, we leverage Graph Neural Network to effectively capture the complex features for the internal scheduling states. Experiments show that by training on small instances, our method can learn scheduling policy that performs well on a wide range of problem scales, which outperforms traditional manual priority rules and state-of-the-art genetic programming based hyper-heuristics.