In this paper, we propose a delay-aware Q-learning-based routing algorithm - XiA - for sending data from users to nearest Access Points (APs) through Unmanned Aerial Vehicle (UAV) swarms communicating using 6G technology. These UAVs assist the ground networks in overcoming communication voids while maneuvering through different demographics. However, the communication links in the THz band have limited transmission range, causing the UAVs to frequently disconnect from the swarm. We overcome such issues by waiting until the UAV comes in contact with others in case of non-time-sensitive data. In the case of time-sensitive data, the UAVs send the data to the APs through Low Earth Orbit (LEO) satellites. To empower XiA to adapt to the changing environments and expensive delays in LEO, we model the rewards by accounting for spreading and absorption in the 6G channels, and Doppler effect and pointing error in the satellite channel. We show our bias for the parameters through extensive simulations and prove that the Q-model in XiA achieves convergence under all conditions. Additionally, in comparison with state-of-the-art solutions, we observe that XiA offers an improved delay of 82%.