This paper studies the stability for a class of systems with time-varying delay subject to controller failure. It explores under what conditions of controller failure the system is still exponentially stable. The concept of controller failure frequency is introduced. First, the delay system with failed controller is modelled as a class of switched delay systems. Next, based on two lemmas developed in this paper and the piecewise Lyapunov functional method, sufficient conditions are provided to guarantee the exponential stability of the system when controller failure occurs, which is in terms of the solvability of a set of linear matrix inequalities (LMIs). Finally, an example of a networked control system is given to illustrate the effectiveness of the proposed method.