The Internet of Vehicles (IoV) system has recently attracted more attention. However, IoV applications require massive computations within strict time limits. Computation energy consumption is also a significant concern in IoV applications. Thus, this paper establishes a vehicle edge computing architecture by combining edge computing and IoV to improve the computation ability of IoV. To optimize the offloading computation process, we model the entire process as a Markov decision process (MDP). Computation delay, computation energy consumption and communication quality are considered in a utility function to establish a multi-objective optimization problem. A deep reinforcement learning algorithm based on a multi-step deep Q network (MSDQN) is proposed to solve the MDP without considering the complicated transmission channels. Especially, the optimal multi-step value is found via experiments. Simulation results show that the proposed offloading algorithm can significantly reduce the IoV computation delay and computation energy consumption in processing computationally intensive tasks.