This paper proposes an efficient model for numerically evaluating the impact of random surface roughness on the internal impedance for large-scale interconnect structures. The effective resistivity (ER) and effective permeability (EP) are numerically formulated to avoid the computationally prohibitive global discretization, while maintaining the model accuracy and flexibility. A modified stochastic integral equation (SIE) method is proposed to significantly speed up the computation for the mean values of ER and EP under the assumption of random surface roughness. Numerical experiments then verify the efficacy of our approach.