The mean square convergence of the kernel least mean square (KLMS) algorithm has been studied in a recent paper [B. Chen, S. Zhao, P. Zhu, J. C. Principe, Mean square convergence analysis of the kernel least mean square algorithm, Signal Processing, vol. 92, pp. 2624-2632, 2012]. In this paper, we continue this study and focus mainly on the initial convergence behavior. Two measures of the convergence performance are considered, namely the weight error power (WEP) and excess mean square error (EMSE). The analytical expressions of the initial decreases of the WEP and EMSE are derived, and several interesting facts about the initial convergence are presented. An illustration example is given to support our observation.