In this paper we develop a whole set of parallel algorithms for improving the computation efficiency of a neurodynamic optimization (NDO) system proposed in our previous work recently. The NDO method is able to solve the sparse signal recovery problems in compressive sensing with the globally convergent optimal solution approximating to the L₀ norm minimization, but has the shortcoming with heavy computation load that is an obstacle for its practical applications. The parallel algorithms are implemented on graphic processing units (GPU) programmed with CUDA language and applied to recovering compressively sensed sparse signals. Experiment results given in the paper show that the new parallel method can improve its computation efficiency significantly with the speedup ratio of more than 60 compared with the original serial NDO algorithm implemented on CPU, while keeping the solution precision unchanged.