Abstract
Because the expense of estimating the optimal value of the relaxation parameter in the successive over-relaxation (SOR) method is usually prohibitive, the parameter is often adaptively controlled. In this paper, new adaptive SOR methods are presented that are applicable to a variety of symmetric positive definite linear systems and do not require additional matrix-vector products when updating the parameter. To this end, we regard the SOR method as an algorithm for minimising a certain objective function, which yields an interpretation of the relaxation parameter as the step size following a certain change of variables. This interpretation enables us to adaptively control the step size based on some line search techniques, such as the Wolfe conditions. Numerical examples demonstrate the favourable behaviour of the proposed methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
A function \(f\in \mathbb {R}^{n}\to \mathbb {R}\) is said to be strictly convex if and only if for all \(\boldsymbol {x},\boldsymbol {y}\in \mathbb {R}^{n}\) (x≠y) and λ ∈ (0, 1), it holds that f(λx + (1 − λ)y) < λf(x) + (1 − λ)f(y).
A function \(f\in \mathbb {R}^{n}\to \mathbb {R}\) is said to be coercive if and only if f(x) →∞ for ∥x∥→∞.
References
Bai, Z.Z., Chi, X.B.: Asymptotically optimal successive overrelaxation methods for systems of linear equations. J. Comput. Math. 21, 603–612 (2003)
Gonzalez, O.: Time integration and discrete Hamiltonian systems. J. Nonlinear Sci. 6, 449–467 (1996). https://doi.org/10.1007/s003329900018
Grimm, V., McLachlan, R.I., McLaren, D., Quispel, G.R.W., Schönlieb, C.B.: Discrete gradient methods for solving variational image regularisation models. J. Phys. A 50, 295201 (2017). https://doi.org/10.1088/1751-8121/aa747c
Hadjidimos, A.: Successive overrelaxation (SOR) and related methods. J. Comput. Appl. Math. 123(1–2), 177–199 (2000). https://doi.org/10.1016/S0377-0427(00)00403-9. Numerical analysis 2000, Vol. III. Linear algebra
Hageman, L.A., Young, D.M.: Applied Iterative Methods. Academic Press, New York (1981)
Hairer, E., Lubich, C.: Energy-diminishing integration of gradient systems. IMA J. Numer. Anal. 34, 452–461 (2014). https://doi.org/10.1093/imanum/drt031
Hairer, E., Wanner, G.: Solving Ordinary Differential Equations II: Stiff and Diffirential-Algebraic Problems. Springer Series in Computational Mathematics, 2nd edn., vol. 14. Springer, Berlin (1996)
Itoh, T., Abe, K.: Hamiltonian-conserving discrete canonical equations based on variational difference quotients. J. Comput. Phys. 76, 85–102 (1988). https://doi.org/10.1016/0021-9991(88)90132-5
Matsuo, T., Furihata, D.: A stabilization of multistep linearly implicit schemes for dissipative systems. J. Comput. Appl. Math. 264, 38–48 (2014). https://doi.org/10.1016/j.cam.2013.12.028
McLachlan, R.I., Quispel, G.R.W., Robidoux, N.: Unified approach to Hamiltonian systems, Poisson systems, gradient systems, and systems with Lyapunov functions or first integrals. Phys. Rev. Lett. 81, 2399–2403 (1998). https://doi.org/10.1103/PhysRevLett.81.2399
McLachlan, R.I., Quispel, G.R.W., Robidoux, N.: Geometric integration using discrete gradients. R. Soc. Lond. Philos. Trans. Ser. A Math. Phys. Eng. Sci. 357, 1021–1045 (1999). https://doi.org/10.1098/rsta.1999.0363
Meng, G.Y.: A practical asymptotical optimal SOR method. Appl. Math. Comput. 242, 707–715 (2014). https://doi.org/10.1016/j.amc.2014.06.034
Miyatake, Y., Sogabe, T., Zhang, S.: On the equivalence between SOR-type methods for linear systems and the discrete gradient methods for gradient systems. J. Comput. Appl. Math. 342, 58–69 (2018). https://doi.org/10.1016/j.cam.2018.04.013
Quispel, G.R.W., McLaren, D.I.: A new class of energy-preserving numerical integration methods. J. Phys. A 41(045), 206 (2008). https://doi.org/10.1088/1751-8113/41/4/045206
Quispel, G.R.W., Turner, G.S.: Discrete gradient methods for solving ODEs numerically while preserving a first integral. J. Phys. A 29, L341–L349 (1996). https://doi.org/10.1088/0305-4470/29/13/006 https://doi.org/10.1088/0305-4470/29/13/006
Ren, L., Ren, F., Wen, R.: A selected method for the optimal parameters of the AOR iteration. J. Inequal. Appl. 2016, 279 (2016). https://doi.org/10.1186/s13660-016-1196-8
Ringholm, T., Lazić, J., Schönlieb, C.B.: Variational image regularization with Euler’s elastica using a discrete gradient scheme. SIAM J. Imaging Sci. 11, 2665–2691 (2018). https://doi.org/10.1137/17M1162354
Varga, R.S.: Matrix Iterative Analysis, 2nd edn. Springer, Berlin (2000)
Acknowledgements
The authors are grateful for various comments by anonymous referees.
Funding
This work has been supported in part by JSPS, Japan KAKENHI Grant Numbers 16K17550, 16KT0016, 17H02829, and 18H05392.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Miyatake, Y., Sogabe, T. & Zhang, SL. Adaptive SOR methods based on the Wolfe conditions. Numer Algor 84, 117–132 (2020). https://doi.org/10.1007/s11075-019-00748-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-019-00748-0