This paper investigates the problem of defending against false data injection (FDI) attacks on power system state estimation. Although many research works have been previously reported on addressing the same problem, most of them made a very strong assumption that some meter measurements can be absolutely protected. To address the problem practically, a reasonable approach is to assume whether or not a meter measurement could be compromised by an adversary does depend on the defense budget deployed by the defender on the meter. From this perspective, our contributions focus on designing the least-budget defense strategy to protect power systems against FDI attacks. In addition, we also extend to investigate choosing which meters to be protected and determining how much defense budget to be deployed on each of these meters. We further formulate the meter selection problem as a mixed integer nonlinear programming problem, which can be efficiently tackled by Benders' decomposition. Finally, extensive simulations are conducted on IEEE test power systems to demonstrate the advantages of the proposed approach in terms of computing time and solution quality, especially for large-scale power systems.