The frequency-domain Kalman filter (FKF) has been utilized in many audio signal processing applications due to its fast convergence speed and robustness. However, the performance of the FKF in under-modeling situations has not been investigated. This letter presents an analysis of the steady-state behavior of the commonly used diagonalized FKF and reveals that it suffers from a biased solution in under-modeling scenarios. An effective improvement of the FKF is proposed, having the benefits of the guaranteed optimal steady-state behavior at the cost of a very limited increase of computational burden. The convergence behavior of the proposed algorithm is also analyzed. Computer simulations are conducted to validate the improved performance of the proposed method.