A control design method is presented for trajectory tracking and rate of penetration control of the directional down-hole drilling process based on a high-order drilling dynamic model. A finite element dynamics model is first customized and further formulated into the optimal control problem; then, the single-neuron adaptive critic dual-heuristic programming (SNAC-DHP)-based controller is constructed in the finite receding horizon. Instead of searching all the states’ and inputs’ spaces, the SNAC-DHP approach uses one neural network (NN) to approximate the gradient of the cost-to-go function to obtain the optimal control inputs. The advantage of using this approach is that the computational expenses will mostly depend on the convergence of the NN rather than the dimension of the system. Therefore, this approach can be used to solve the optimal control problem of the high-dimensional system, such as the directional drilling system, in a computationally efficient fashion. To this end, a series of simulations are conducted to evaluate the efficacy of this proposed controller. The robustness of this controller is also discussed, and the low computational expenses also show the potential to apply this approach to other high-dimensional control problems.