This paper studies optimal day-ahead scheduling of a battery swapping and charging system (BSCS) for electric vehicles (EVs) from a new perspective of multiple decision makers. It is considered that the BSCS locally incorporates the battery swapping and charging processes, and the two processes are managed by two operators, called a battery swapping operator (BSO) and a battery charging operator (BCO), respectively. Our main contribution is to propose a bilevel model where the BSO acts as the leader to receive and serve the battery swapping requests from EV users, and the BCO acts as the follower to interact with the grid and control battery charging and discharging power. We reformulate the bilevel optimization problem into an equivalent single-level problem that is a nonconvex mixed-integer nonlinear program (MINLP), and its size can easily become very large. To solve the problem efficiently, we develop a new heuristic composed of two parts, i.e., an estimation of the integer solution and an algorithm based on the alternating direction method (ADM). The results show that the proposed heuristic performs well in solving large-scale problems, providing close-to-optimal solutions quickly. In addition, compared to a social welfare maximization model that follows most existing related works, the proposed bilevel model can increase the number of swapped-out batteries by 35% and the batteries’ average energy state by 6%, improving the quality of battery swapping services.