This paper tackles the joint user association (UA) and power control (PC) problem for dynamic heterogeneous networks (HetNets), where the number of active users varies over time. Since the joint UA and PC problem is a mixed-integer problem, it is challenging to develop a joint UA and PC algorithm that closely achieves the optimal proportional fairness (PF) performance while satisfying the tight computation time budget of recent 3GPP standards. Although previous studies have reduced the computation time on the UA and PC by employing a fully-connected-neural network (FCN), the FCN architecture is not generally applicable to dynamic HetNets because of a varying number of users. In pursuit of scalable neural network design, we propose an unsupervised learning-based UA and PC algorithm using a recurrent neural network (RNN). Notably, in the RNN-based UA, we relax the combinatorial UA variable into a low-dimensional continuous variable, the length of which is invariant to the number of users. The optimality loss in the variable relaxation is upper-bounded by the negligibly small duality gap. Through extensive experiments, we show that the proposed scheme has a 10% higher PF performance than conventional optimization-based methods with the same number of flops and closely achieves the optimal PF performance for various numbers of users.