Reducing the number of multiplikations in iterative processes

Summary

In any iteration scheme, such as v ^k=f(Qv ^k−1), where a fixed matrix multiplies a vector that depends on the iteration number, Winograd's method for computing inner products can be used in a straightforward manner to reduce the number of multiplications required at the cost of more additions. The key observation is that certain quantities required by Winograd's method have to be computed only at the first iteration. In the Jacobi method for solving systems of linear equations, f is linear. Gauss-Seidel iteration often converges faster than Jacobi iteration, but it cannot be put in the above form. A simple trick is necessary to apply Winograd's method in an efficient recursive manner. Our proposed method is better than the naive method when it is faster to add than to multiply. Versions of Jacobi and Gauss-Seidel iteration appropriate for optimization (as in Markov decision problems) are presented. The analysis specializes easily to the linear equation case.