Digital Twin-Edge Network (DTEN) architecture is emerging as a critical component in the landscape of 6G networks, offering the promise of real-time data processing, system simulation, and edge-cloud computing. The integration of unmanned aerial vehicles (UAVs) and high-altitude platform systems (HAPS) within these architectures further adds to the complexity and capabilities, particularly in time-sensitive scenarios. The study of delay-sensitive queue-aware task offloading of real-time applications in such intricate dynamic energy-constrained networks remains nascent. This paper aims to bridge this gap by exploring optimizing IoT device association, offloading decisions, and resource allocation to maximize energy efficiency (EE) in an Air-to-Ground DTEN (A2G-DTEN). Our primary objective is to maximize the EE of the network while adhering to constraints related to queuing delays, maximum permissible task latency, and computing capabilities of edge servers. We proposed a comprehensive problem formulation and offered solutions leveraging a deep deterministic policy gradient (DDPG) based algorithm with two other baselines. The numerical results show that our proposed DDPG-based algorithm achieves high EE despite the strict task delay constraints.