Online approximation of the optimal path for a control affine nonlinear autonomous agent subject to input and state constraints (e.g., actuator saturation, obstacles, no-enter zones) is considered. A model-based adaptive dynamic programming technique is implemented to locally estimate the unknown value function associated with the optimal path-planning problem. By performing a local approximation, the locations of the static obstacles do not need to be known until the obstacles are within a defined approximation window. The developed feedback policy guarantees ultimately bounded convergence of the approximated path to the optimal path without the requirement of persistence of excitation, typically required for online adaptive dynamic programming. Simulation results are presented to illustrate the performance of the proposed method.