In this paper, energy efficiency (EE)-oriented partial spectrum reuse (PSR) scheme for heterogeneous cellular networks (HCN) with wired backhaul is studied with the aid of stochastic geometry theory. We first derive the total expected delay for a typical user by taking into account retransmissions over the wireless link, as well as the backhaul delay incurred from wired backhaul. Furthermore, we minimize the whole delay experienced by the users from the perspective of small cell density and the minimization of the total energy cost then can be addressed relying on the optimal small cell density and PSR factor. The main takeaway is that if we account for the backhaul, it can significantly reverse the relationship between the total network energy cost and the delay experienced by the users with varying signal to interference ratio (SIR) threshold. Theoretical analysis results indicate that, unlike previous known conclusions, sacrificing delay cannot always be traded for energy saving, and the feasible conditions under which the optimal energy-delay tradeoff exists are also obtained. Moreover, the optimal pair of PSR factor and SIR threshold to achieve the optimal energy-delay tradeoff is also derived. Both the analytical and simulation results show that significant energy saving can be achieved through tolerable sacrifice of delay performance with careful designs according to our analysis.