In this technical article, to seek the optimal solution to time-dependent nonlinear optimization subject to linear inequality and equality constraints (TDNO-IEC), the gradient-based differential neural-solution, termed as GDN model, is proposed and researched. Notably, TDNO-IEC is first converted into the nonhomogeneous linear equation with the dynamic parameter. Second, differential neural-solution with the aid of gradient is designed. The contrastive theoretical analyses among the GDN model, gradient-based neural network (GNN), and the dual neural network (DNN) prove that the proposed GDN model has higher accuracy for eliminating the large solution error with exponential convergence. In addition, reasonable convergent time of the GDN model is guaranteed by activation functions with simple formulation. Last, an illustrative example and real-world applications, including robot motion planning and data dimension reduction and reconstruction, further validate the high availability of the proposed GDN model.