The majority studies on resource allocation are based on continuous power allocation. However, the transmit power can be assumed as a finite set of discrete power levels only, especially for practical digital cellular systems. By simply rounding up or down, the conventional continuous power allocation algorithm will not suffice. In this paper, via transmitting signal on discrete power levels, multi-cell power allocation is modeled as a combinatorial optimization problem, which proved to be NP-hard. Ant colony optimization (ACO) is applied to get near-optimal solution of the problem. In particular, a multiple power level based searching graph is designed, and conventional ACO is improved that ant colonies in each cell cooperate to maximize system throughput. Simulation results indicate that with only 4 power levels, the proposed algorithm can achieve a significant rate gain over the existing schemes.