Model parameters' estimation is one of the most important tasks in the analysis and design process of a nonlinear dynamic system in real time, especially in the presence of noise. This article presents a novel approach in estimating important parameters of gray-box model for such a system on real nonlinear EEG to simulate efficiently the dynamic characteristics of neurons. Specifically, the proposed methodology exploits unscented Kalman filter (UKF) that is combined with chaos neural population model to formulate the interaction between the cortical areas. The proposed methodology is compared with the state-of-the-art parameters' estimation techniques to verify the efficiency of the UKF on the gray-box model. Experimental results show that the proposed method demonstrates the lowest error value of root mean square error (RMSE) among existing parameter estimation methods. The robustness of the proposed approach is further validated in its convergence and automation, with minimum error relatively than others and without any user-specified input, respectively.