In response to the challenges of spectrum scarcity and the exponential growth of the number of connected devices, this paper addresses the joint optimization problem of user-base station association, channel assignment and power allocation in a multi-band wireless network, where sub-6 GHz, millimeter wave, and terahertz frequency bands coexist. The problem is formulated as a mixed integer non-linear programming, a known NP-hard problem. Each user requests both a minimum data rate and a minimum reliability level defined by a signal-to-noise ratio. Considering the goal of optimizing the number of satisfied users, this paper proposes a multi-agent deep reinforcement learning solution. Simulation results convincingly demonstrate the effectiveness of our proposed algorithm and its ability to learn fast the best resource allocation solution.