Multi-radio access technologies (RATs) networks, where various heterogeneous networks (HetNets) coexist, are in service nowadays and considered a main enabling technology for future networks. In such networks, managing radio resources is challenge. In this letter, we address the problem of RATs-edge devices (EDs) association and joint power and bandwidth allocation in multi-RAT multi-homing HetNets. The problem is formulated as mixed-integer non-linear programming, whose objective is to cost-effectively maximize the network constrained sum-rate. Due to the high complexity of the problem, we propose a multi-agent deep reinforcement learning (DRL)-based scheme to solve it. Simulation results show that our proposed scheme efficiently learns the optimal policy and enhances the network sum-rate by 80.95% compared to key benchmarks.