As the extensive application of massive multiple-input multiple-output (MIMO) in 5G and beyond 5G (B5G) networks, multi-user (MU) MIMO scheduling faces big challenges on performance enhancement with effective interference coordination and computational complexity reduction. Plenty of deep learning and reinforcement learning for wireless resource scheduling are proposed to solve the above issues via a well trained network, instead of executing iteration search on each scheduling period. However, the dimension of the channel state information and the size of user combination set may increase exponentially in massive MIMO system, which makes the neural network over complicated and causes severe convergent issues. In this paper, a novel Actor-Critic framework is developed to overcome the above existing issues for the single-cell downlink multi-user scheduling issue in massive MIMO system. Pointer network is investigated as the policy network in our proposed algorithm, which transfers the complicated selection issue among user combinations to a user sequential selection issue based on conditional probability. Simulation results show that the performance of our method is very close to that of the greedy algorithm with much less computational complexity. Moreover, our proposal is robust and effective with the increase of the number of antennas and users.