In this paper, dynamic subcarrier and power allocation problem is considered in the context of asymptotic utility maximization in multi-carrier systems. Using gradient-based resource allocation, we formulate an optimization problem involving subcarrier and transmit power allocation for each time slot, and propose an optimal algorithm solving the problem. Since the optimal algorithm is impractical due to its complexity, a simple suboptimal algorithm is also proposed based on generalized Benders decomposition. Furthermore, we identify the performance of equal power allocation policy by showing that equal power allocation is not always near optimal in general utility maximization problem and characterizing the optimality condition of equal power allocation. This result not only justifies the use of our dynamic power allocation, but also generalizes the performance of equal power allocation, which is well-known to be approximately optimal to the sum capacity maximization problem in high SNR (signal-to-noise ratio) regime. Our algorithms and analysis are verified through extensive simulations.