As a good example of Advanced Driver-Assistance Systems (ADAS), Advisory Speed Assistance (ASA) helps improve driving safety and possibly energy efficiency by showing advisory speed to the driver of an intelligent vehicle. However, driver-based speed tracking errors often emerge, due to the perception and reaction delay, as well as imperfect vehicle control, degrading the effectiveness of ASA system. In this study, we propose a learning-based approach to modeling driver behavior, aiming to predict and compensate for the speed tracking errors in real time. Subject drivers are first classified into different types according to their driving behaviors using the k-nearest neighbors (k-NN) algorithm. A nonlinear autoregressive (NAR) neural network is then adopted to predict the speed tracking errors generated by each driver. A specific traffic scenario has been created in a Unity game engine-based driving simulator platform, where ASA system provides advisory driving speed to the driver via a head-up display (HUD). A human-in-the-loop simulation study is conducted by 17 volunteer drivers, revealing a 53% reduction in the speed error variance and a 3% reduction in the energy consumption with the compensation of the speed tracking errors. The results are further validated by a field implementation with a real passenger vehicle.