The consideration of reliability in controller design is able to avoid the potential actuator faults from inappropriate strategies. This work presents an optimal reliability-critical controller to avoid potential actuator faults by developing a probabilistic adaptive dynamic programming (ADP) algorithm with the estimation of fault interruption. The proposed algorithm distinguishes from existing ADPs in that the structural reliability is considered in policy iteration, endowing the resultant controller with the capacity to avoid potential actuator faults. The algorithm relaxes the generalized damage energy-Hamilton–Jacobi–Bellman equation to a reliability-critical problem, which is solved by proposing a probabilistic policy iteration method. Instead of studying the stability regardless of physical damage, the effect of physical damage is considered in the system stability in the form of structural reliability, and the probabilistic policy iteration guarantees the optimal relation between the stability and structural reliability. Finally, the effectiveness of the proposed algorithm is verified by conducting experimental tests.