Parallel model combination (PMC) techniques have been successfully and popularly used in noisy speech recognition. Previous PMC methods such as the log-normal PMC and the log-add PMC generally use approximate compensation for delta parameters. In this paper we propose a PMC scheme with improved delta parameter compensation (PMC-IDC), which employs the log-normal PMC method for the static distribution compensation and a new delta parameter compensation. By using this new PMC-IDC scheme, improvement over the other PMC methods can be obtained in different additive noises at the expense of slight increase in computational complexity.