Multi-Objective Optimization (MOO) has always been an important issue in the field of wireless communications. With the development of 5G networks, more objectives have been concerned to improve the user experience. The relationship between these multiple objectives is complex or even conflicting, which increases the difficulty of solving the MOO problems. Traditional multi-objective optimization algorithms (e.g., genetic algorithm) have higher computation complexity and require to store multiple models for the preference of different objectives. Therefore, in this paper, a multi-objective scheduling model based on the Actor-Critic framework is proposed, which can effectively solve the multi-user scheduling problem under Massive Multiple-Input Multiple-Output (MIMO), and utilize a single model to approximate the Pareto frontier. In the single-cell downlink scheduling scenario, the proposed model is applied to the two objective optimization, i.e., channel capacity and fairness. The simulation results show that the performance of our model is close to the theoretical optimal value in the single-objective case. The Pareto frontier can be uniformly approximated in the multi-objective case, and it has strong robustness to never-seen preference combinations.