Under high penetration of renewable energy resources, more and more reactive power control devices are integrated into power grids. The large-scale deployment of these devices requires upgrading existing reactive power control solutions. To improve energy efficiency and voltage profiles of the power grids under different operating conditions, this paper proposes a fully distributed multiple agent system based optimal reactive control solution. To update its control setting, a reactive controller only needs information measured locally or obtained from its neighboring buses. The updating rules of the subgradient based algorithm are derived under mild assumptions. The solution is able to provide comparable steady state performance as that of centralized optimization solutions. Due to the simplicity of communication topology and the reduced amount of data to process, the solution can provide timely response to changes of operating conditions. Simulation studies with power systems of different sizes demonstrate the effectiveness of proposed control solution.