In this paper, we propose a hardware friendly On-Chip Training Algorithm for the memristive Neuromorphic circuits (OCTAN). Although the proposed algorithm has a simple hardware like that of the random weight change (RWC) algorithm, it is much more efficient in terms of convergence speed and accuracy. In this algorithm, weights of the circuit are updated individually by a small value and the effect of individual weight update is assessed. If the weight change causes an increase in the error of the network, the weight update is reversed by applying the same change in the reverse direction twice. The usefulness of the proposed algorithm is verified by training some neuromorphic circuits for different applications. Compared to RWC and stochastic least-mean-squares (SLMS) training algorithms, our proposed algorithm needs, on average, $329\times$ fewer epochs to find the minimum error point. Moreover, the accuracy of the networks trained by OCTAN is, on average, about 46% higher than those of RWC and SLMS algorithms. Additionally, a hardware for OCTAN is presented. This hardware provides a speedup of $172\times$ ( $61\times$ ) compared to that of the RWC (SLMS) algorithm. Finally, the impact of PVT (process, voltage, and temperature) variations is studied on the proposed training hardware indicating an average training error increase of less than 3.27% in the presence of variations.