The ability of unmanned aerial vehicles (UAVs) to communicate and cooperate among themselves in an aerial swarm network is of paramount importance for collaborative missions. It enables the swarm to function as a collective unit to plan trajectories, share tasks, coordinate flight route for collision avoidance or to optimize a given mission objective. In this work, a cellular radio interface known as "sidelink" is investigated for establishing efficient UAV-to-UAV (U2U) multi-hop communication within the aerial swarm network. First, we look into the salient features of cellular sidelink (PC5 radio interface) as a key enabler of U2U communication, and then present a multi-hop network model design to extend cellular connectivity services to out-of-coverage UAVs during missions. Using extensive simulations, we evaluate and study the performance of sidelink-assisted multi-hop U2U communication model in terms of the packet delivery ratio (PDR) for different key scheduling parameters of mode-4 sensing-based autonomous resource selection.