Autonomous Vehicles promise to transport people in a safer, accessible, and even efficient way. Nowadays, real-world autonomous vehicles are build by large teams from big companies with a tremendous amount of engineering effort. Deep Reinforcement Learning can be used instead, without domain experts, to learn end-to-end driving policies. Here, we combine Curriculum Learning with deep reinforcement learning, in order to learn without any prior domain knowledge, an end-to-end competitive driving policy for the CARLA autonomous driving simulator. To our knowledge, this is the first work which provides consistent results of our driving policy on all the town scenarios provided by CARLA. Moreover, we point out two important issues in reinforcement learning: the former is about learning the value function in a stable way, whereas the latter is related to normalizing the learned advantage function. A proposal of a solution to these problems is provided.