Armed and autonomous unmanned aerial vehicle (UAV) swarms are a new type of aerial threat due to their numerical superiority and cooperative communication, and existing countermeasures cannot completely eliminate whole swarms. In this letter, we design an algorithm based on deep reinforcement learning called GCPDDQN to find the optimal attack sequence for large-scale UAV swarm, so as to achieve the purpose of decomposing the network into small pieces and destroying swarm communications. Numerical simulations show that GCPDDQN can speed up the collapse of the network using only the simplest features and network architectures which are changeable to adjust to different scenarios.