We propose a handover (HO) strategy that helps minimize energy consumption at base stations (BSs) in heterogeneous networks. To conserve energy at BSs, the proposed strategy is to uniquely consider energy consumption generated during the HO execution phase under an expectation on the stochastic behavior of HO parameters in making an HO decision. This plays a critical role in eliminating unnecessary HOs occurring within the areas covered by multiple BSs. To verify the energy-saving potential of the proposed strategy, we formulate an HO problem as a constrained Markov decision process in which the objective is to minimize the expected total energy consumption at BSs in serving a traffic flow, and the constraints are a diverse set of delay requirements for supporting multiple traffic classes. Simulation results show the achievable performance gain through the proposed strategy from the perspective of energy efficiency.