The fast moving characteristics of high-speed trains pose a challenge to the beam alignment of high-speed railway millimeter wave communication systems. With powerful learning capabilities, machine learning-based methods can help improve the beam alignment performance, such as greatly reducing the delay. In this paper, a non-convex optimization problem is formulated aiming at maximizing the received power of downlink transmission, and deep reinforcement learning is used to assist beam alignment. Particularly, an adaptive beam alignment algorithm based on prioritized experience replay double deep Q-Learning is proposed to adjust beam direction dynamically. The algorithm observes the position of the train and the beam direction of the train roof mobile relay to guide the beam adjustment. To reduce the adjustment frequency and the requirements for train position accuracy, the service range of remote radio head is divided into multiple location bins. The beam direction adjustment is only executed when the train enters the next location bin. Simulation results verify that compared with other baseline schemes, the proposed algorithm can effectively improve the received power and reduce the beam alignment delay.