In this letter, we focus on the subcarrier allocation problem for device-to-device (D2D) communication in cellular networks to improve the cellular energy efficiency (EE). Our goal is to maximize the weighted cellular EE and its solution is obtained by using a game-theoretic learning approach. Specifically, we propose a lower bound instead of the original optimization objective on the basis of the proven property that the gap goes to zero as the number of transmitting antennas increases. Moreover, we prove that an exact potential game applies to the subcarrier allocation problem and it exists the best Nash equilibrium (NE) which is the optimal solution to optimize the lower bound. To find the best NE point, a distributed learning algorithm is proposed and then is proved that it can converge to the best NE. Finally, numerical results verify the effectiveness of the proposed scheme.