This paper investigates the problem of age of information (AoI) aware radio resource management for a platooning system. Multiple autonomous platoons exploit the cellular wireless vehicle-to-everything (C-V2X) communication technology to disseminate the cooperative awareness messages (CAMs) to their followers while ensuring timely delivery of safety-critical messages to the Road-Side Unit (RSU). Due to the challenges of dynamic channel conditions, centralized resource management schemes that require global information are inefficient and lead to large signaling overheads. Hence, we exploit a distributed resource allocation framework based on multi-agent reinforcement learning (MARL), where each platoon leader (PL) acts as an agent and interacts with the environment to learn its optimal policy. Existing RL algorithms consider a holistic reward function for the group's collective success, which often ends up with unsatisfactory results and cannot obtain an optimal policy for each agent. Consequently, we modify the MARL framework in a way that each agent learns to find an optimal policy to improve its individual reward according to its actions and observations while cooperating with other agents to learn a global team reward. Numerical results indicate our proposed algorithm's effectiveness compared with the conventional RL methods applied in this area.