Leakage terms are introduced in certain adaptive filters to improve stability and robustness to transmission errors. The effects of that modification to the conventional procedure is analyzed for the gradient technique. Estimations are given for the resulting bias of the coefficients and error signal power increase. As an illustration, a prediction filter in an ADPCM codec is considered.