This paper proposes an inverse reinforcement learning (IRL)-based method to optimize power allocation for multiuser cellular networks. An optimization problem is formulated to maximize the achievable sum information rate of all receivers. In contrast to traditional reinforcement learning (RL)-based methods, the proposed IRL-based one does not require to design the reward function manually, which is able to determine the reward function efficiently and automatically from the expert policy. The weighted minimum mean square error (WMMSE) method is used to serve as an expert policy to obtain the reward function, and the action space and state space are designed. Simulation results show that the proposed IRL-based method achieves about 99 % of the sum information rate achieved by the pure WMMSE method, but the running time of the proposed IRL-based one is about 1/19 of that required of pure WMMSE method.