This paper focuses on the design of a low order robust stabilizer for the tracking/disturbance rejection problem based on the internal model principle in the time varying setting. The existing stabilization approaches are either lack of robustness or results in a high order design, which limits the potential for broad application. The method proposed in this paper overcomes this bottleneck by taking advantage of the unique structure of the time varying internal model based control system. Instead of using a dynamic stabilizer with high order, this approach uses a sequence of time varying gains that are injected into the internal model unit. A critical issue addressed is how to avoid the non-convex optimization associated with the time varying gain synthesis and then convert the stabilizer design into a series of convex Linear Matrix Inequality (LMI) constraints. The approach is then validated on an experimental system and demonstrated to be robust and computationally efficient.