In this manuscript, we propose a joint channel and power assignment algorithm for an unmanned aerial vehicle (UAV) swarm communication system based on multi-agent deep reinforcement learning (DRL). Regarded as an agent, each UAV to UAV (U2U) link can choose the optimal channel and power according to the current situation after training is successfully completed. Further, a mixing network is introduced based on DRL, where Q values of every single agent are non-linearly mapped, and we call it the QMIX algorithm. As it accesses state information, QMIX can learn to enrich the joint action value function. The proposed method can be used for both unicast and multicast scenarios. Experiments show that each U2U link can be trained to meet the constraints of UAV communication and minimize the interference to the system. For unicast communication, the communication rate is increased up to 15.6% and 8.9% using the proposed DRL method compared with the well-known random and adaptive methods, respectively. For multicast communication, the communication rate is increased up to 6.7% using the proposed QMIX method compared with the DRL method and 13.6% using DRL method compared with adaptive method. Besides, the successful transmission probability can maintain a high level.