Ultra reliable and low latency communication (uRLLC) is of extreme importance in vehicular networks. To ensure the stringent quality of service of uRLLC in vehicular networks, this paper proposes a deep reinforcement learning-based power allocation scheme. Specifically, we formulate the power allocation problem to maximize the long-term averaged system capacity subject to the system reliability and latency constraints considering the finite blocklength constraint. It is analyzed that the problem is non-convex and intractable by the traditional optimization methods. To deal with the problem, we resort to a deep reinforcement learning (DRL) algorithm, which is widely referred as the deep deterministic policy gradient (DDPG) method, to learn the optimal policy with imperfect instantaneous channel state information (CSI). Simulation results reveal that our proposed DDPG-based power allocation algorithm can not only increase the long-term averaged system capacity but also greatly enhance the latency and reliability performance, compared with the traditional DRL-based power allocation algorithms.