This paper studies the problem of actuator or controller failure of a class of time-varying delay systems. It is assumed once the actuator or controller fails, the system becomes unstable. It addresses how long and how frequent the failure is so that the system can keep exponentially stable. Based on a switching method, this kind of failure problems is converted into one of finding a switching signal to stabilize the corresponding switched delay system. Then using average dwell time method, the sufficient conditions guaranteeing exponential stability are derived in the case of actuator or controller failure. A numerical example is used to show the effectiveness of the proposed method.