For systems with variable gain, proportional plus integral controllers need to be tuned in the highest gain state of the system. This can lead to sluggish response when the system is in a low gain state. Previous research has shown that error modulated state setting control can be used to improve the transient response of variable gain systems. Error modulated state setting controllers smoothly combine steady state predictions with a proportional plus integral controller. In the research presented here online learning was incorporated into error modulated state setting control. Rather than adjusting the steady state predictions, the agent learned how aggressively to use the predictions by creating a map from the input space of the steady state predictions to the performance of the controller. A nonlinear second order system was tested in simulation. The error modulated state setting controller with online learning outperformed a well tuned PI controller on a combined tracking and disturbance rejection task. The average system tracking error was reduced by 34%.