This paper proposes a disaster relief service (DRS) method in hierarchical non-terrestrial networks (HNTNs). We aim to maximize the number of satisfactory connections (NSCs) by optimizing the unmanned aerial vehicles (UAV) radio resources, computing resources, and trajectory at each time instant. In particular, the UAVs are exploited as aerial base stations (ABSs) to provide a channel for the reduced capability (RedCap) user equipment (UE) based on power domain non-orthogonal multiple access (PD-NOMA). While the disaster condition leads to nonoperational terrestrial networks, sending data in the shortest time is crucial for the RedCap UEs monitoring critical infrastructures. Hence, the end-to-end (E2E) delay is the quality of service (QoS) constraint. The underline problem is solved based on a multi-agent recurrent deterministic policy gradient (MARDPG) algorithm, in which the ABSs cooperate to maximize the NSCs and find their optimal policy interacting with the environment. We further consider the experience-sharing module (ESM) for the agents that encode actions and observations using long short-term memory (LSTM), which leads to each agent utilizing other agents’ history of actions and observations. Numerical results illustrate the effect of different parameters, such as the number of subcarriers, users, and the maximum tolerable E2E delay on the proposed solution. Besides, the outcomes demonstrate the superiority of our proposed algorithm compared to the other existing works from literature.