Energy conservation is a critical issue in the emerging standard IEEE 802.16e/m WiMAX supporting mobility. To guarantee QoS requirements in real-time services such as VoIP, traditional energy saving strategies adopt constant listen-sleep intervals, ignoring the On-Off characteristics of human speech. However, statistically the silence period can account for nearly 60% of the whole speech in time scale. Therefore, neglecting this fact can lead to unnecessary periodical listening in the silence duration, and, in turn, can result in excessive waste of battery energy. In this paper, we adopt a hybrid energy management for the downlink simplex VoIP. We also give an evaluation model to analyze the performance of the scheme. Guided by this model, we obtain the optimal window adjustment parameters. Extensive simulation results have validated the analytical model, and indicated that, compared with the traditional scheme, the hybrid scheme can achieve as much as 90% reduction in energy dissipation during silence period, while meeting the QoS requirements satisfactorily at the same time.