The field of artificial intelligence is advancing rapidly, with reinforcement learning making significant strides in solving various sequential decision problems in machine learning. As research progresses, multi-agent reinforcement learning has emerged in the field of reinforcement learning and has been applied to numerous domains. Vehicle formation is an important means of transportation for reducing vehicle energy consumption and improving air quality. Controlling sparse vehicles on the road to form formations is a fascinating research topic. In this paper, we propose a planning framework for formation control based on federated learning and multi-agent reinforcement learning to address this problem. We model vehicle energy consumption to accurately assess energy usage during vehicle formation. Additionally, we incorporate reinforcement learning algorithms into the vehicle formation process to enable multi-vehicle asynchronous decision-making and save formation time. We also introduce federated learning into the training process to significantly reduce overall system communication.