The use of heterogeneous small cell-based networks to offload the traffic of existing cellular systems has recently attracted significant attention. One main challenge is solving the joint problems of user association, resource allocation, and interference mitigation. The goal of this paper is to design a self-organizing algorithm that can solve these problems, simultaneously. To this end, this joint resource allocation problem is formulated as an optimization problem which is then solved using log-sum-exp approximation. This solution is then shown to require complete information of the whole network which is not scalable with the network size. To address this scalability issue, a novel Markov chain approach is proposed and its transition probabilities are shown to eventually converge to the near optimal solution without complete information. Furthermore, the gap between the optimal and converged solutions is shown to be bounded. Simulation results show that our proposed algorithm effectively offloads the traffic from macro-cell base station to small-cell base stations. Moreover, the results also show that this algorithm converges very quickly independent of the number of possible configurations.