A new identification algorithm for nonlinear singularly perturbed system using multi-time scales recurrent high-order neural networks is proposed in this paper. The high-order neural networks have simple structure and strong nonlinear approximation capability, which enables it to model the nonlinear singularly perturbed systems more accurately with less computation complexity, compared to multilayer neural networks. The optimal bounded ellipsoid algorithm, which is originally designed for discrete time systems, is introduced to update the weights of continuous multi-time scales neural networks. Compared to other widely used gradient-like updating methods, the on-line identification algorithm proposed in this paper can realize faster convergence, due to the adaptive “learning rate” of the weights updating laws. The effectiveness of the proposed scheme is demonstrated by simulation results.