Internet of Vehicles (IoV) is an emerging technology that has been rapidly developing in recent years, which combines the internet, wireless communication technology, and vehicle electronics to realize the intelligence of interaction between vehicles and greatly improve the efficiency and safety of the transportation system. However, high-frequency and cyclical communications between vehicles and the limited system capacity in the region have led to serious conflicts in the allocation of wireless resources for vehicle networking. To solve this problem, this paper models resource allocation as a multi-agent deep reinforcement learning problem and proposes a Multi-Agent Actor-Critic (MAAC) algorithm based on distributed execution. The multi-agent algorithm can achieve distributed computing by modeling the training mechanism and reward function, thus improving resource utilization. Experimental results show that the proposed algorithm improves the total Vehicle-to-Infrastructure (V2I) link capacity and Vehicle-to-Vehicle (V2V) link transmission success rate.