We design decentralized frequency control of multi-area power systems that will re-balance power and drive frequencies to their nominal values after a disturbance. Both generators and controllable loads are utilized to achieve frequency stability while minimizing regulation cost. In contrast to recent results, the design is completely decentralized and does not require communication between areas. Our control enforces operational constraints not only in equilibrium but also during transient. Moreover, our control is capable of adapting to unknown load disturbance. We show that the closed-loop system is asymptotically stable and converges to an equilibrium that minimizes the regulation cost. We present simulation results to demonstrate the effectiveness of our design.